-- Logs begin at Tue 2022-07-19 19:49:48 UTC, end at Tue 2022-07-19 21:25:51 UTC. -- Jul 19 20:39:04.961488 dv-cinder-749155-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 19 20:39:04.969911 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 19 20:39:04 2022] *** Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: nodename: dv-cinder-749155-8-iscsi Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: machine: x86_64 Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: clock source: unix Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: pcre jit disabled Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: detected number of CPU cores: 4 Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: current working directory: / Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: detected binary path: /usr/bin/uwsgi-core Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: your processes number limit is 47680 Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: your memory page size is 4096 bytes Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: detected max file descriptor number: 2048 Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: lock engine: pthread robust mutexes Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: thunder lock: enabled Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 19 20:39:04.981128 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 19 20:39:05.019440 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: Python main interpreter initialized at 0x55d5dd7299c0 Jul 19 20:39:05.019440 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: python threads support enabled Jul 19 20:39:05.019440 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: your server socket listen backlog is limited to 100 connections Jul 19 20:39:05.019440 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: your mercy for graceful operations on workers is 90 seconds Jul 19 20:39:05.019832 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: mapped 403077 bytes (393 KB) for 2 cores Jul 19 20:39:05.019897 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: *** Operational MODE: preforking *** Jul 19 20:39:05.020183 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: *** uWSGI is running in multiple interpreter mode *** Jul 19 20:39:05.020183 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: spawned uWSGI master process (pid: 86656) Jul 19 20:39:05.020422 dv-cinder-749155-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 19 20:39:05.023637 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: spawned uWSGI worker 1 (pid: 86668, cores: 1) Jul 19 20:39:05.023637 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: spawned uWSGI worker 2 (pid: 86669, cores: 1) Jul 19 20:39:05.023637 dv-cinder-749155-8-iscsi devstack@keystone.service[86656]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 19 20:39:10.556151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 20:39:10.556151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: from cryptography.utils import int_from_bytes Jul 19 20:39:10.556935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 20:39:10.556935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: from cryptography.utils import int_from_bytes Jul 19 20:39:10.559834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 20:39:10.559834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: from cryptography.utils import int_from_bytes Jul 19 20:39:10.560300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 20:39:10.560300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: from cryptography.utils import int_from_bytes Jul 19 20:39:10.605350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 20:39:10.605530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 20:39:10.605715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] command line args: [] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 20:39:10.606084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 20:39:10.606238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ================================================================================ {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 20:39:10.606501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] admin_token = **** {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.606796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] config_dir = [] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.606982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.607166 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] config_source = [] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.607334 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] debug = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.607524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.609253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.609363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 20:39:10.609437 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.609503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 20:39:10.609606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.609675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] command line args: [] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 20:39:10.609733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.609795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 20:39:10.609795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ================================================================================ {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 20:39:10.609920 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] list_limit = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.609920 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] log_config_append = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.610058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] admin_token = **** {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.610122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.610180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] config_dir = [] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.610243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] log_dir = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.610307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.610392 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] log_file = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.610497 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] config_source = [] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.610602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.610719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] debug = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.610838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.610955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611345 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] list_limit = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611655 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] log_config_append = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.611946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] log_dir = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.612071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.612135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] log_file = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.612200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.612259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.612315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.612864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.613052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.613245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.613429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.613562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.613698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.613839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.613839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.613998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.614098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.614213 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] publish_errors = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.614348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.614427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.614496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.614573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.614658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.614775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.614775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.614969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.615044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.615144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.615219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] standard_threads = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.615330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.615450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.615530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.615642 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.615716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.615834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.615834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] use_journal = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.615834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] use_json = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.616137 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] use_stderr = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.616774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.616916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] use_syslog = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.616991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] publish_errors = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.617106 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.617186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.617305 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.617387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.617387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.617567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.617686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.617767 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.617885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.617970 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.618084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] standard_threads = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.618158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.618255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.618336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.618505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.618595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.618667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.618787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.618863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] use_journal = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.619007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] use_json = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.619090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.619212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] use_stderr = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.619294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.619370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] use_syslog = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.619484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.619558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 20:39:10.619711 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.619798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.619952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.620034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.620184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.620259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.620448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.620541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.620692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.620775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.620925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.621001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.621175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.621263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.621423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.621499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.621663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.621739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.621894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.622051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.622130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.622200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.622359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.622436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.622606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.622693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.622840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.622916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.623075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.623150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.623302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.623385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.623530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.623605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.623761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.623837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.624004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.624081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.624248 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.624378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.624501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.624577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.624729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.624807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.625011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.625142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.625222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.625294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.625430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.625514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.625682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.625758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.625913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.625995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.626143 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.626224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.626373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.626448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.626598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.626672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.626825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.626909 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.627050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.627131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.627275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.627351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.627501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.627606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.627745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.627853 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.627971 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.628089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.628201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.628356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.628455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.628608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.628683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.628846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.628931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.629102 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.629215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.629338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.629442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.629568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.629681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.629800 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.629902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.630030 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.630139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.630263 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.630392 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.630493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.630570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.630719 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.630795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.630959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.631049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.631172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.631251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.631408 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.631487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.631635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.631712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.631915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.631993 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.632143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.632221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.632386 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.632478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.632621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.632700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.632853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.632935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.633098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.633223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.633349 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.633436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.633575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.633656 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.633819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.633897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.634052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.634133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.634285 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.634365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.634516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.634593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.634756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.634841 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.634988 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.635065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.635219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.635299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.635454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.635531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.635684 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.635758 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.635915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.635991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.636145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.636223 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.636402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.636482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.636633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.636710 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.636891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.636974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.637193 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.637269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.637437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.637506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.637686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.637765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.637926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.638012 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.638205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.638290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.638414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.638490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.638605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.638682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.638832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.638908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.639076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.639154 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.639311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.639404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.639506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.639589 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.639721 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.639834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.639957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.640075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.640195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.640315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.640462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.640580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.640691 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.640812 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.640946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.641058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.641201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.641286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.641440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.641517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.641670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] auth.external = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.641749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.641898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.641986 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.642188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.642265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.642416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.642532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.642670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] auth.password = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.642803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.642923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] auth.token = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.643030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.643141 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.643259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.643366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.643491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.643610 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.643778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.643911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.644043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.644171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.644299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.644484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.644617 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.644748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.644876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] auth.external = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.644989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] credential.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.645097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.645207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.645421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.645421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.645664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.645790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] auth.password = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.645900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.646016 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] auth.token = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.646158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.646286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.646415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.646546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.646675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.646805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.646946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.647074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.647203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.647820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.647952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.648079 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.648215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.648385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.648496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.648606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] credential.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.648720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.648843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.648843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.649062 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.649184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.649297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.649384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.649446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.649515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.649587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] federation.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.649658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.649728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.649832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.649912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.650072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.650150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.650304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.650440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.650570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.650711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.650847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.650972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.651096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.651222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.651358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.651484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.651621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.651750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.651877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.652007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.652108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.652230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] federation.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.652511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.652641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.652766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.652892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.653017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.653125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.653236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.653350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.653429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.653501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.653501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.653713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.653789 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.653861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.653931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.654002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.654174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.654318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.654448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.654576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.654706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.654833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.654962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.655107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.655107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.655281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.655342 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.655401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.655579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.655671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.655827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.655905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.656080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.656156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.656313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.656479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.656587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.656663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.656839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.656968 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.657142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.657272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.657368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.657465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.657631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.657710 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.657927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.658045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.658174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.658442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.658519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.658708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.658883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.658975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.659202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.659323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.659474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.659741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.659818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.659981 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.660195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.660274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.660460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.660669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.660747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.660915 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.661092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.661170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.661383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.661521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.661630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.661863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.661984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.662107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.662342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.662419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.662577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.662816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.662895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.663135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.663830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.663945 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.664100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.664257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.664356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.664580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.664717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.664816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.665061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.665152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.665298 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.665533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.665618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.665780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.666000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.666085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.666245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.666432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.666510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.666718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.666878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.666957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.667188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.667343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.667427 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.667643 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.667774 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.667880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.668118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.668195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.668404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.668636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.668726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.668882 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.669075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.669153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.669357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.669435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.669589 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.669668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.669817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.669896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.670051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.670130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.670285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.670364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.670517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.670595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.670746 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.670824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.670991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.671071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.671244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.671324 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.671487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.671564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.671720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.671797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.671950 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.672027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.672175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.672252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.672430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.672509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.672666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.672742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.672894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.672971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.673125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.673203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.673357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.673434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.673597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.673674 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.673846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.673926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.674088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.674166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.674314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.674391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.674555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.674634 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.674788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.674865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.675018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.675096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.675250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.675328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.675477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.675555 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.675718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.675795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.675962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.676041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.676196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.676273 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.676470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.676550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.676701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.676780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] resource.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.676954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.677032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.677188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.677266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.677420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.677498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.677653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.677731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.677896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.677973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.678128 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.678205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.678356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.678432 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.678616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.678693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.678854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] resource.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.678933 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.679086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] role.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.679162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.679333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.679409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] role.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.679561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.679634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.679813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.679895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.680059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.680137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.680289 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.680388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.680556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.680635 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.681116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.681383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.681646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.681893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.682124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.682389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.682655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.682903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.683146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.683505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.683625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.683852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.683967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.684130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.684225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] role.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.684393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.684498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] role.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.684706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.685034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.685165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.685315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.685513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.685620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.685817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.685921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.686119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.686300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.686399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.686563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.686689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.686839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.686931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.687094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.687173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.687356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.687457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.687603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.687842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.688091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.688360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.688616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.688864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.689293 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.689530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.689779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.690026 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.690276 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.690525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.690760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.691014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.691264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.691503 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.691730 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.691971 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.692202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.692475 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.692729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.692975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.693208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.693440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.693667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.693751 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.693906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] token.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.694158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.694242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.694412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.694647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.694728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.694903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.694983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.695149 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.695226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] token.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.695384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.695487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.695623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.695863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.695940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.696107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.696193 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.696368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.696452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.696596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.696743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.696821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.697064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.697180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.697308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.697476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.697630 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.697740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.698064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.698142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.698441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.698517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.698706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.connection = **** {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.698794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.698954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.699041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.699201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.699280 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.699438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.699604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.699816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.699963 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.700064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.700227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.700305 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.700550 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.700627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.700879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.700975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.connection = **** {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.701059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.701253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.701332 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.701493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.701596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.701724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.701848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.701967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.702134 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.702211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.702425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.702549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.702636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.702840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.702919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.703089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.703167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.703260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86668) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 20:39:10.703345 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.703785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.704092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.704367 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.704620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.705022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.705369 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.705627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.705869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.706120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.706462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 20:39:10.706642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86669) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 20:39:10.735090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.735317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.735572 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.739600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.739842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.740072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.743913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.749622 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.772092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.772857 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.778443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.778716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.916969 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.922232 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.968594 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.968842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.971050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:10.971344 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.035485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.035709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.035929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.036144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.036383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.036599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.036683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.036837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.037063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.037285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.037505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.037723 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.061933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.062181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.062409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.062618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.062838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.062927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.063052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.063135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.063263 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.063420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.063721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.063949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.064169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.064414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.074882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.076498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.088280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.088533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.088740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.088957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.089170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.089254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.089379 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.089465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.089588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86668) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.089673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.089861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.090080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.090296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.090513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86669) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 20:39:11.106693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.106898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 20:39:11.107233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.107430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 20:39:11.107695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.107950 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.108062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.108202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.108301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 20:39:11.108484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.108665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.108947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 20:39:11.109223 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.109486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.109749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.110370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.119498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.119771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.120037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.120527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.122768 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.123046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.123357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.123711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.128517 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.132286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.132588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.132867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.132990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.133254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.133532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.133792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.134059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.134332 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.134599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.134872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.135139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.136348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.136632 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.136912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.137210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.137470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.137729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.137996 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.138267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.138535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.138805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.139072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.162503 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.164154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.164580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.169742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.169866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.170003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.170278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.171593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.171875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.177513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.177626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.177787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.177874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.178141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.182672 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.182955 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.183230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.183506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.183775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.184139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.184440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.184705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.185974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.186273 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.191114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.191440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.191809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.192096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.192398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.192679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.193094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.193320 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.203912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.204195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.208803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.213163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.213452 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.215575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.215873 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.218453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.220970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.222078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.224831 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.225130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.225925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.226254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.226541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.226820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.227109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.227384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.230220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.233979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.238092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.238391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.238779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.239113 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.239397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.239677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.246203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.246640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.247011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.247169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.256222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.258509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.258779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.262012 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.262304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.262478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.262635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.262767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.262854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.263025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.263108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.263271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.271784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.274087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.274365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.277091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.277370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.277457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.277679 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.277782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.277992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.278098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.278270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.278355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.278541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.278668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.278819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.293554 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.293912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.294200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.294498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.294818 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.295191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.295476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.298320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.301635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.305087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.306928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.308577 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.308854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.309131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.316560 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.317423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.318161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.321285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.321915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.322209 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.322488 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.322758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.324624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.326244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.328015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.328517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.328811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.332004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.332287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.332581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.337581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.339430 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.341064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86668) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.341338 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.341613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.341955 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.342228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.342498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.342771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.343033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.343120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.343306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.343427 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.343568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.343717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.343837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.344002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.344082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.344367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.344643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86668) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.354433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.354716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.355057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.363532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86669) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 20:39:11.363810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.364091 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.364395 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.364669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.364950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.365231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.365528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.365890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.366184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.366553 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.366819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.367089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.367366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86669) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 20:39:11.382890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d5dd7299c0 pid: 86668 (default app) Jul 19 20:39:11.386321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638864e-17c9-4336-ac48-7b3018bf06db None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:11.386608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638864e-17c9-4336-ac48-7b3018bf06db None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:11.386893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638864e-17c9-4336-ac48-7b3018bf06db None None] PATH_INFO: `/v3` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:11.391205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1/1] 10.222.0.85 () {50 vars in 791 bytes} [Tue Jul 19 20:39:11 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:11.410362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d5dd7299c0 pid: 86669 (default app) Jul 19 20:39:19.308709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bcccf-0d38-4a7b-978f-7cf850a59d9e None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:19.309273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bcccf-0d38-4a7b-978f-7cf850a59d9e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:19.309273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bcccf-0d38-4a7b-978f-7cf850a59d9e None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:19.310591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 2/2] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:39:19.318836 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5b6ea-737e-42c4-98c6-25ef5ff41a2e None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:19.320164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5b6ea-737e-42c4-98c6-25ef5ff41a2e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:19.320164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5b6ea-737e-42c4-98c6-25ef5ff41a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:19.388846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: INFO dbcounter [None req-a4f5b6ea-737e-42c4-98c6-25ef5ff41a2e None None] Registered counter for database keystone Jul 19 20:39:19.413179 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG oslo_db.sqlalchemy.engines [None req-a4f5b6ea-737e-42c4-98c6-25ef5ff41a2e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86669) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 20:39:19.419643 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writer thread running {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 20:39:19.601202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG passlib.handlers.bcrypt [None req-a4f5b6ea-737e-42c4-98c6-25ef5ff41a2e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86669) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 19 20:39:19.601526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG passlib.handlers.bcrypt [None req-a4f5b6ea-737e-42c4-98c6-25ef5ff41a2e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86669) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 19 20:39:19.658318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-a4f5b6ea-737e-42c4-98c6-25ef5ff41a2e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:19.659965 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a4f5b6ea-737e-42c4-98c6-25ef5ff41a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:19.791434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1/3] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 19 20:39:19.973509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bcccf-0d38-4a7b-978f-7cf850a59d9e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:20.044493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: INFO dbcounter [None req-a95bcccf-0d38-4a7b-978f-7cf850a59d9e None None] Registered counter for database keystone Jul 19 20:39:20.064465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG oslo_db.sqlalchemy.engines [None req-a95bcccf-0d38-4a7b-978f-7cf850a59d9e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86668) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 20:39:20.070743 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writer thread running {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 20:39:20.189640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bcccf-0d38-4a7b-978f-7cf850a59d9e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:20.193956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71e7c2-8081-464c-91b8-12a4d21ffd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:20.194655 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e7c2-8081-464c-91b8-12a4d21ffd5c admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:20.194942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e7c2-8081-464c-91b8-12a4d21ffd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:20.195225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71e7c2-8081-464c-91b8-12a4d21ffd5c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:20.203661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71e7c2-8081-464c-91b8-12a4d21ffd5c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:20.256819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cd71e7c2-8081-464c-91b8-12a4d21ffd5c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 19 20:39:20.257106 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71e7c2-8081-464c-91b8-12a4d21ffd5c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:20.267385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 3/4] 10.222.0.85 () {58 vars in 1324 bytes} [Tue Jul 19 20:39:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:39:20.273407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5b6ea-737e-42c4-98c6-25ef5ff41a2e None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:20.288511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5b6ea-737e-42c4-98c6-25ef5ff41a2e None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:20.292980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba276f-82d6-4348-8e9f-ecbb3874ae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:20.294093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba276f-82d6-4348-8e9f-ecbb3874ae18 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:20.294374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba276f-82d6-4348-8e9f-ecbb3874ae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:20.294649 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba276f-82d6-4348-8e9f-ecbb3874ae18 admin admin] PATH_INFO: `/v3/projects/3f322b7a5256429ea77db2c08c676fad` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:20.298695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba276f-82d6-4348-8e9f-ecbb3874ae18 admin admin] RBAC: Authorizing `identity:get_project(project_id=3f322b7a5256429ea77db2c08c676fad)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:20.349978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: WARNING keystone.common.rbac_enforcer.enforcer [None req-28ba276f-82d6-4348-8e9f-ecbb3874ae18 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 19 20:39:20.351054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba276f-82d6-4348-8e9f-ecbb3874ae18 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:20.355444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 2/5] 10.222.0.85 () {56 vars in 1207 bytes} [Tue Jul 19 20:39:20 2022] GET /identity/v3/projects/3f322b7a5256429ea77db2c08c676fad => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:22.435180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b499d-fede-44a8-ba63-a292884c5563 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:22.435719 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b499d-fede-44a8-ba63-a292884c5563 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:22.435823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b499d-fede-44a8-ba63-a292884c5563 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:22.440255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 4/6] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:22.446311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b837-ba33-47dc-a80a-6ad5abc78c6c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:22.446747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b837-ba33-47dc-a80a-6ad5abc78c6c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:22.446848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b837-ba33-47dc-a80a-6ad5abc78c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:22.466260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d3c7b837-ba33-47dc-a80a-6ad5abc78c6c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:22.467743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d3c7b837-ba33-47dc-a80a-6ad5abc78c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:22.475608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 3/7] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 19 20:39:22.497693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b499d-fede-44a8-ba63-a292884c5563 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:22.510368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b499d-fede-44a8-ba63-a292884c5563 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:22.514635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbaf333-250e-4e30-9250-1973ae913ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:22.515268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaf333-250e-4e30-9250-1973ae913ead admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:22.515554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaf333-250e-4e30-9250-1973ae913ead admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:22.515833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaf333-250e-4e30-9250-1973ae913ead admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:22.524264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbaf333-250e-4e30-9250-1973ae913ead admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:22.526079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbaf333-250e-4e30-9250-1973ae913ead admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:22.535685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 5/8] 10.222.0.85 () {58 vars in 1324 bytes} [Tue Jul 19 20:39:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:39:22.541708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7b837-ba33-47dc-a80a-6ad5abc78c6c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:22.550729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7b837-ba33-47dc-a80a-6ad5abc78c6c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:22.555769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe7408-f02d-4a10-bf90-1ba2913c40db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:22.556916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe7408-f02d-4a10-bf90-1ba2913c40db admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:22.557198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe7408-f02d-4a10-bf90-1ba2913c40db admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:22.557470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe7408-f02d-4a10-bf90-1ba2913c40db admin admin] PATH_INFO: `/v3/users/adee0d7d3c614774bcfa96ea7aed77e6` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:22.558872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe7408-f02d-4a10-bf90-1ba2913c40db admin admin] RBAC: Authorizing `identity:get_user(user_id=adee0d7d3c614774bcfa96ea7aed77e6)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:22.560722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe7408-f02d-4a10-bf90-1ba2913c40db admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:22.562275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 4/9] 10.222.0.85 () {56 vars in 1198 bytes} [Tue Jul 19 20:39:22 2022] GET /identity/v3/users/adee0d7d3c614774bcfa96ea7aed77e6 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:24.660409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b806ad5-af28-451e-b730-44676b480772 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:24.660921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b806ad5-af28-451e-b730-44676b480772 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:24.661009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b806ad5-af28-451e-b730-44676b480772 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:24.662319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 6/10] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:24.668087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a64316-5368-4a3c-b1b1-739c8d68f11a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:24.668696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a64316-5368-4a3c-b1b1-739c8d68f11a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:24.668838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a64316-5368-4a3c-b1b1-739c8d68f11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:24.685594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-53a64316-5368-4a3c-b1b1-739c8d68f11a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:24.686784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-53a64316-5368-4a3c-b1b1-739c8d68f11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:24.712198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281eda5-4074-4b1d-b8b6-811e7981b744 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:24.712763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281eda5-4074-4b1d-b8b6-811e7981b744 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:24.713258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281eda5-4074-4b1d-b8b6-811e7981b744 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:24.714812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 7/11] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:24.722087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd26cb-e82a-4860-8653-4213b4850f8e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:24.722526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd26cb-e82a-4860-8653-4213b4850f8e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:24.722943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd26cb-e82a-4860-8653-4213b4850f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:24.773791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG passlib.handlers.bcrypt [None req-98cd26cb-e82a-4860-8653-4213b4850f8e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86668) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 19 20:39:24.774208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG passlib.handlers.bcrypt [None req-98cd26cb-e82a-4860-8653-4213b4850f8e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86668) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 19 20:39:24.775954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 5/12] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:24.800172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a64316-5368-4a3c-b1b1-739c8d68f11a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:24.822504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-98cd26cb-e82a-4860-8653-4213b4850f8e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:24.824453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-98cd26cb-e82a-4860-8653-4213b4850f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:24.824798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:24.906842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a64316-5368-4a3c-b1b1-739c8d68f11a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:24.934844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 8/13] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:39:24.943880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3c19f-d951-48d0-bb16-6eb67b74c25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:24.945355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3c19f-d951-48d0-bb16-6eb67b74c25d None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:24.945790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3c19f-d951-48d0-bb16-6eb67b74c25d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:24.946132 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3c19f-d951-48d0-bb16-6eb67b74c25d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:24.958195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3c19f-d951-48d0-bb16-6eb67b74c25d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:24.958392 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd26cb-e82a-4860-8653-4213b4850f8e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-65d3c19f-d951-48d0-bb16-6eb67b74c25d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:24.969482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:24.970578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:24.971515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 6/14] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:39:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:39:24.978904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3c19f-d951-48d0-bb16-6eb67b74c25d None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:25.063974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:25.083812 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3c19f-d951-48d0-bb16-6eb67b74c25d None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:25.084585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd26cb-e82a-4860-8653-4213b4850f8e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:25.090390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:25.129723 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8c2fc-e7b0-4d34-8332-88ec2469f73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:25.130393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8c2fc-e7b0-4d34-8332-88ec2469f73e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:25.130677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8c2fc-e7b0-4d34-8332-88ec2469f73e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:25.130954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8c2fc-e7b0-4d34-8332-88ec2469f73e None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:25.131749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8c2fc-e7b0-4d34-8332-88ec2469f73e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:25.132584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8c2fc-e7b0-4d34-8332-88ec2469f73e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:25.133120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124de777-f84d-4d3b-b0a3-5b7d19fc1f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:25.133990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8c2fc-e7b0-4d34-8332-88ec2469f73e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:25.134145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124de777-f84d-4d3b-b0a3-5b7d19fc1f08 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:25.134524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124de777-f84d-4d3b-b0a3-5b7d19fc1f08 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:25.134884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124de777-f84d-4d3b-b0a3-5b7d19fc1f08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:25.146209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 7/15] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:25.151872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8c2fc-e7b0-4d34-8332-88ec2469f73e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:25.208889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124de777-f84d-4d3b-b0a3-5b7d19fc1f08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:25.210864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124de777-f84d-4d3b-b0a3-5b7d19fc1f08 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:25.264973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8c2fc-e7b0-4d34-8332-88ec2469f73e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:25.298719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136e347-a5ea-4683-bd70-a9a72175300c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:25.299593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136e347-a5ea-4683-bd70-a9a72175300c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:25.299876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136e347-a5ea-4683-bd70-a9a72175300c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:25.300152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136e347-a5ea-4683-bd70-a9a72175300c None admin] PATH_INFO: `/v3/users/adee0d7d3c614774bcfa96ea7aed77e6` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:25.301471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136e347-a5ea-4683-bd70-a9a72175300c None admin] RBAC: Authorizing `identity:get_user(user_id=adee0d7d3c614774bcfa96ea7aed77e6)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:25.302985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136e347-a5ea-4683-bd70-a9a72175300c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:25.304650 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 8/16] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:25 2022] GET /identity/v3/users/adee0d7d3c614774bcfa96ea7aed77e6 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:25.311239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136e347-a5ea-4683-bd70-a9a72175300c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:25.318798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 9/17] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:39:25.325093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124de777-f84d-4d3b-b0a3-5b7d19fc1f08 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:25.327084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:25.328153 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:25.420764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136e347-a5ea-4683-bd70-a9a72175300c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:25.433718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124de777-f84d-4d3b-b0a3-5b7d19fc1f08 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:25.466556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8c7ef-cbce-4902-a707-fc56a6ee4358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:25.467280 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c7ef-cbce-4902-a707-fc56a6ee4358 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:25.467568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c7ef-cbce-4902-a707-fc56a6ee4358 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:25.467850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c7ef-cbce-4902-a707-fc56a6ee4358 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:25.469112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8c7ef-cbce-4902-a707-fc56a6ee4358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:25.470610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8c7ef-cbce-4902-a707-fc56a6ee4358 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:25.472477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 9/18] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:25.476903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9157d-ae98-4930-a983-47e5e9eb93a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:25.477806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9157d-ae98-4930-a983-47e5e9eb93a3 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:25.478155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9157d-ae98-4930-a983-47e5e9eb93a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:25.478351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8c7ef-cbce-4902-a707-fc56a6ee4358 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:25.478544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9157d-ae98-4930-a983-47e5e9eb93a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:25.497541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9157d-ae98-4930-a983-47e5e9eb93a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:25.499017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9157d-ae98-4930-a983-47e5e9eb93a3 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-53e9157d-ae98-4930-a983-47e5e9eb93a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:39:25.521309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:25.522753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:25.523557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:39:25.523557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:39:25.523557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:39:25.523557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:39:25.523557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 10/19] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:39:25.529327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9157d-ae98-4930-a983-47e5e9eb93a3 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:25.562964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:25.584348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8c7ef-cbce-4902-a707-fc56a6ee4358 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:25.591735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:25.624101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d6f07-edbd-4a2e-a701-20a9156ea1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:25.624858 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d6f07-edbd-4a2e-a701-20a9156ea1c6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:25.625142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d6f07-edbd-4a2e-a701-20a9156ea1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:25.625751 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d6f07-edbd-4a2e-a701-20a9156ea1c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:25.626389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d6f07-edbd-4a2e-a701-20a9156ea1c6 None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.domain.id=default, user.id=adee0d7d3c614774bcfa96ea7aed77e6) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:25.626706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d6f07-edbd-4a2e-a701-20a9156ea1c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:25.628238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d6f07-edbd-4a2e-a701-20a9156ea1c6 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:25.635847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 10/20] 10.222.0.85 () {56 vars in 1322 bytes} [Tue Jul 19 20:39:25 2022] GET /identity/v3/role_assignments?user.id=adee0d7d3c614774bcfa96ea7aed77e6&scope.domain.id=default&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:25.638553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9157d-ae98-4930-a983-47e5e9eb93a3 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:25.677186 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbcaad-aea8-4c32-93e6-a39cf699b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:25.677891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbcaad-aea8-4c32-93e6-a39cf699b866 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:25.678707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbcaad-aea8-4c32-93e6-a39cf699b866 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:25.679004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbcaad-aea8-4c32-93e6-a39cf699b866 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:25.679751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbcaad-aea8-4c32-93e6-a39cf699b866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:25.680079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbcaad-aea8-4c32-93e6-a39cf699b866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:25.681888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbcaad-aea8-4c32-93e6-a39cf699b866 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:25.702872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 11/21] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:28.032462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcc194-112a-4755-a73a-f9848c663e79 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:28.033334 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcc194-112a-4755-a73a-f9848c663e79 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:28.033334 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcc194-112a-4755-a73a-f9848c663e79 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:28.034676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 11/22] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:28.049514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cacf47-a0b7-40ea-8bff-35381821de77 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:28.050093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cacf47-a0b7-40ea-8bff-35381821de77 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:28.050205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cacf47-a0b7-40ea-8bff-35381821de77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:28.075419 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc98f2-53b2-4acd-a9b5-f349da024232 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:28.075752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc98f2-53b2-4acd-a9b5-f349da024232 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:28.075980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc98f2-53b2-4acd-a9b5-f349da024232 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:28.077894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 12/23] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:28.083762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30662f62-3bed-4b36-a8eb-70fa19194d80 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:28.084176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30662f62-3bed-4b36-a8eb-70fa19194d80 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:28.085168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30662f62-3bed-4b36-a8eb-70fa19194d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:28.091589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e5cacf47-a0b7-40ea-8bff-35381821de77 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:28.093239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e5cacf47-a0b7-40ea-8bff-35381821de77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:28.120924 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-30662f62-3bed-4b36-a8eb-70fa19194d80 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:28.122182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-30662f62-3bed-4b36-a8eb-70fa19194d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:28.242832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 12/24] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:28.262745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 13/25] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:39:28.278312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cacf47-a0b7-40ea-8bff-35381821de77 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:28.304098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30662f62-3bed-4b36-a8eb-70fa19194d80 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:28.354465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:28.442047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:28.499267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cacf47-a0b7-40ea-8bff-35381821de77 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:28.501863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30662f62-3bed-4b36-a8eb-70fa19194d80 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:28.577185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917caac6-13fd-40ab-aedd-9d4a0af5aa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:28.577377 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7348c93-145d-4e2e-87d3-bd1b7dbfdf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:28.577915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7348c93-145d-4e2e-87d3-bd1b7dbfdf90 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:28.578373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7348c93-145d-4e2e-87d3-bd1b7dbfdf90 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:28.578661 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917caac6-13fd-40ab-aedd-9d4a0af5aa73 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:28.578803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7348c93-145d-4e2e-87d3-bd1b7dbfdf90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:28.579224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917caac6-13fd-40ab-aedd-9d4a0af5aa73 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:28.579648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917caac6-13fd-40ab-aedd-9d4a0af5aa73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:28.598387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7348c93-145d-4e2e-87d3-bd1b7dbfdf90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:28.600766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7348c93-145d-4e2e-87d3-bd1b7dbfdf90 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:28.609475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917caac6-13fd-40ab-aedd-9d4a0af5aa73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-917caac6-13fd-40ab-aedd-9d4a0af5aa73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:28.624354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:28.625279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:28.626053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 14/26] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:39:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:39:28.629528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917caac6-13fd-40ab-aedd-9d4a0af5aa73 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-e7348c93-145d-4e2e-87d3-bd1b7dbfdf90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:39:28.639245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:28.640102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:28.640965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:39:28.640965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:39:28.640965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:39:28.640965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:39:28.642908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 13/27] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:28.649493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7348c93-145d-4e2e-87d3-bd1b7dbfdf90 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:28.798759 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917caac6-13fd-40ab-aedd-9d4a0af5aa73 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:28.860827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:28.864944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7348c93-145d-4e2e-87d3-bd1b7dbfdf90 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:28.878453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05cf73-53a2-4d2b-b597-afdbb55b9719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:28.879511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05cf73-53a2-4d2b-b597-afdbb55b9719 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:28.880006 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05cf73-53a2-4d2b-b597-afdbb55b9719 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:28.880522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05cf73-53a2-4d2b-b597-afdbb55b9719 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:28.881521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05cf73-53a2-4d2b-b597-afdbb55b9719 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:28.882022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05cf73-53a2-4d2b-b597-afdbb55b9719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:28.886198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05cf73-53a2-4d2b-b597-afdbb55b9719 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:28.888655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:28.902969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 15/28] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:28.904420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34fd0a-2da6-4724-8b1a-bf25476d973e None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:28.904712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34fd0a-2da6-4724-8b1a-bf25476d973e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:28.904987 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34fd0a-2da6-4724-8b1a-bf25476d973e None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:28.906241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 16/29] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:28.911501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34fd0a-2da6-4724-8b1a-bf25476d973e None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:28.924366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205c8b8-903e-4761-ad73-73481636862f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:28.924366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205c8b8-903e-4761-ad73-73481636862f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:28.924366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205c8b8-903e-4761-ad73-73481636862f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:28.924366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205c8b8-903e-4761-ad73-73481636862f None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:28.924908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205c8b8-903e-4761-ad73-73481636862f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:28.925283 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205c8b8-903e-4761-ad73-73481636862f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:28.927307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205c8b8-903e-4761-ad73-73481636862f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:28.964291 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 14/30] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:28.965878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890fc9c-0bee-45d1-9aa2-57775b9b6ef2 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:28.966173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890fc9c-0bee-45d1-9aa2-57775b9b6ef2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:28.966453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890fc9c-0bee-45d1-9aa2-57775b9b6ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:29.015383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e890fc9c-0bee-45d1-9aa2-57775b9b6ef2 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:29.018762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e890fc9c-0bee-45d1-9aa2-57775b9b6ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:29.095555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 15/31] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:29.096826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34fd0a-2da6-4724-8b1a-bf25476d973e None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:29.097448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890fc9c-0bee-45d1-9aa2-57775b9b6ef2 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:29.135124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca30ea-759f-4f63-9372-8e2e6437988b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:29.136380 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca30ea-759f-4f63-9372-8e2e6437988b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:29.137200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca30ea-759f-4f63-9372-8e2e6437988b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:29.137672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca30ea-759f-4f63-9372-8e2e6437988b None admin] PATH_INFO: `/v3/users/adee0d7d3c614774bcfa96ea7aed77e6` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:29.139168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca30ea-759f-4f63-9372-8e2e6437988b None admin] RBAC: Authorizing `identity:get_user(user_id=adee0d7d3c614774bcfa96ea7aed77e6)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:29.140893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca30ea-759f-4f63-9372-8e2e6437988b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:29.142741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 17/32] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:28 2022] GET /identity/v3/users/adee0d7d3c614774bcfa96ea7aed77e6 => generated 247 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:29.144244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42f2b9-ea43-4a2d-a993-938969697395 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:29.144686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42f2b9-ea43-4a2d-a993-938969697395 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:29.145168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42f2b9-ea43-4a2d-a993-938969697395 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:29.146808 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 18/33] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:29.147891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42f2b9-ea43-4a2d-a993-938969697395 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:29.204016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:29.219302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890fc9c-0bee-45d1-9aa2-57775b9b6ef2 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:29.233582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:29.260202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6926c7c-94ba-4af5-a430-db0fd86f216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:29.261337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6926c7c-94ba-4af5-a430-db0fd86f216b None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:29.261829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6926c7c-94ba-4af5-a430-db0fd86f216b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:29.262322 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6926c7c-94ba-4af5-a430-db0fd86f216b None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:29.263506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6926c7c-94ba-4af5-a430-db0fd86f216b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:29.265323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6926c7c-94ba-4af5-a430-db0fd86f216b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:29.281133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42f2b9-ea43-4a2d-a993-938969697395 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:29.320535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fcb6c-1508-41a8-866d-5c8205e9f6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:29.321447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fcb6c-1508-41a8-866d-5c8205e9f6d0 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:29.321887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fcb6c-1508-41a8-866d-5c8205e9f6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:29.322296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fcb6c-1508-41a8-866d-5c8205e9f6d0 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:29.323182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fcb6c-1508-41a8-866d-5c8205e9f6d0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:29.325074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fcb6c-1508-41a8-866d-5c8205e9f6d0 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:29.422137 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 16/34] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:39:29 2022] POST /identity/v3/projects => generated 301 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:39:29.423360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6926c7c-94ba-4af5-a430-db0fd86f216b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:29.468574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 19/35] 10.222.0.85 () {60 vars in 1131 bytes} [Tue Jul 19 20:39:29 2022] POST /identity/v3/roles => generated 225 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:39:29.471236 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b823b47-7dae-4296-aac6-721552c81ee7 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:29.471724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b823b47-7dae-4296-aac6-721552c81ee7 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:29.472289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b823b47-7dae-4296-aac6-721552c81ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:29.514281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3b823b47-7dae-4296-aac6-721552c81ee7 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:29.517290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3b823b47-7dae-4296-aac6-721552c81ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:29.547398 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6926c7c-94ba-4af5-a430-db0fd86f216b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:29.590312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc674d1-309d-4629-a42a-c76389d11ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:29.591640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc674d1-309d-4629-a42a-c76389d11ec7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:29.592250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc674d1-309d-4629-a42a-c76389d11ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:29.593067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc674d1-309d-4629-a42a-c76389d11ec7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:29.594775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc674d1-309d-4629-a42a-c76389d11ec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:29.597714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc674d1-309d-4629-a42a-c76389d11ec7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:29.601513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 17/36] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:29.609192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc674d1-309d-4629-a42a-c76389d11ec7 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:29.628784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 20/37] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:29.632534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:29.666058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b823b47-7dae-4296-aac6-721552c81ee7 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:29.744830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:29.806932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b823b47-7dae-4296-aac6-721552c81ee7 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:29.845096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc674d1-309d-4629-a42a-c76389d11ec7 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:29.903331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e71113-69a5-4aee-bc24-1ac1dd56dfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:29.904245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e71113-69a5-4aee-bc24-1ac1dd56dfb6 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:29.904564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e71113-69a5-4aee-bc24-1ac1dd56dfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:29.904847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e71113-69a5-4aee-bc24-1ac1dd56dfb6 None admin] PATH_INFO: `/v3/domains/default/users/adee0d7d3c614774bcfa96ea7aed77e6/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:29.907553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e71113-69a5-4aee-bc24-1ac1dd56dfb6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=adee0d7d3c614774bcfa96ea7aed77e6, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:29.910231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e71113-69a5-4aee-bc24-1ac1dd56dfb6 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:29.931966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 18/38] 10.222.0.85 () {58 vars in 1361 bytes} [Tue Jul 19 20:39:29 2022] PUT /identity/v3/domains/default/users/adee0d7d3c614774bcfa96ea7aed77e6/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:39:29.933884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a4e09-25e1-4c7d-8ee6-21135064b0f5 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:29.934245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a4e09-25e1-4c7d-8ee6-21135064b0f5 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:29.934529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a4e09-25e1-4c7d-8ee6-21135064b0f5 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:29.936065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 19/39] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:29.942476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9563ad-112a-4632-ae4b-1354807ad062 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:29.942894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9563ad-112a-4632-ae4b-1354807ad062 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:29.943278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9563ad-112a-4632-ae4b-1354807ad062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:29.966186 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-7c9563ad-112a-4632-ae4b-1354807ad062 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:29.968112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7c9563ad-112a-4632-ae4b-1354807ad062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:29.987234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1480710-d204-4890-9710-f0a32698c122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:29.987963 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1480710-d204-4890-9710-f0a32698c122 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:29.988311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1480710-d204-4890-9710-f0a32698c122 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:29.988699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1480710-d204-4890-9710-f0a32698c122 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:29.989598 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1480710-d204-4890-9710-f0a32698c122 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:29.991515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1480710-d204-4890-9710-f0a32698c122 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:30.011176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 21/40] 10.222.0.85 () {60 vars in 1131 bytes} [Tue Jul 19 20:39:29 2022] POST /identity/v3/roles => generated 219 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:39:30.012623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdf846-ae43-4b15-b3fc-8147dee15876 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:30.012925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdf846-ae43-4b15-b3fc-8147dee15876 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:30.013209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdf846-ae43-4b15-b3fc-8147dee15876 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:30.014530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 22/41] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:30.020588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027d2c0-7475-4dab-9868-5dcfa65bf72f None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:30.020888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027d2c0-7475-4dab-9868-5dcfa65bf72f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:30.021175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027d2c0-7475-4dab-9868-5dcfa65bf72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:30.047420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-f027d2c0-7475-4dab-9868-5dcfa65bf72f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:30.049007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f027d2c0-7475-4dab-9868-5dcfa65bf72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:30.062993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:30.076777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 20/42] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:39:30.103903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9563ad-112a-4632-ae4b-1354807ad062 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:30.151794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 23/43] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:39:30.177278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027d2c0-7475-4dab-9868-5dcfa65bf72f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:30.245536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:30.276186 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9563ad-112a-4632-ae4b-1354807ad062 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:30.312351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027d2c0-7475-4dab-9868-5dcfa65bf72f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:30.339422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9c990-84b6-4ff2-9c56-09a24f1d10fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:30.340825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9c990-84b6-4ff2-9c56-09a24f1d10fe None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:30.341336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9c990-84b6-4ff2-9c56-09a24f1d10fe None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:30.341854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9c990-84b6-4ff2-9c56-09a24f1d10fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:30.344312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9c990-84b6-4ff2-9c56-09a24f1d10fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:30.347086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9c990-84b6-4ff2-9c56-09a24f1d10fe None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:30.350036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840143e9-90e2-41a5-a7a7-668bd14d2c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:30.350943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840143e9-90e2-41a5-a7a7-668bd14d2c64 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:30.351405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840143e9-90e2-41a5-a7a7-668bd14d2c64 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:30.351922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840143e9-90e2-41a5-a7a7-668bd14d2c64 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:30.352757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 21/44] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:30.353229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840143e9-90e2-41a5-a7a7-668bd14d2c64 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:30.354894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840143e9-90e2-41a5-a7a7-668bd14d2c64 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:30.359024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9c990-84b6-4ff2-9c56-09a24f1d10fe None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:30.371692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 24/45] 10.222.0.85 () {60 vars in 1131 bytes} [Tue Jul 19 20:39:30 2022] POST /identity/v3/roles => generated 223 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:39:30.449708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:30.497898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9c990-84b6-4ff2-9c56-09a24f1d10fe None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:30.548295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b6ce8-b681-4e4a-aed4-4b59690c7d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:30.548946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b6ce8-b681-4e4a-aed4-4b59690c7d67 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:30.549237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b6ce8-b681-4e4a-aed4-4b59690c7d67 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:30.549523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b6ce8-b681-4e4a-aed4-4b59690c7d67 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:30.550252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b6ce8-b681-4e4a-aed4-4b59690c7d67 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:30.551724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b6ce8-b681-4e4a-aed4-4b59690c7d67 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:30.689879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 22/46] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:39:30 2022] POST /identity/v3/projects => generated 312 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:39:32.214301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92c87-6da7-411f-b270-f2697ab81f54 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:32.214847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92c87-6da7-411f-b270-f2697ab81f54 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:32.214847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e92c87-6da7-411f-b270-f2697ab81f54 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:32.216239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 25/47] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:32.221713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976a96e-4e88-4b69-bd66-5c1c243675a4 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:32.222090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976a96e-4e88-4b69-bd66-5c1c243675a4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:32.222281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976a96e-4e88-4b69-bd66-5c1c243675a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:32.239362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e976a96e-4e88-4b69-bd66-5c1c243675a4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:32.240692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e976a96e-4e88-4b69-bd66-5c1c243675a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:32.307456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 23/48] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:32.330317 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e92c87-6da7-411f-b270-f2697ab81f54 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:32.442308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e92c87-6da7-411f-b270-f2697ab81f54 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:32.445286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:32.479783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a812c-fe32-4e97-bf04-a88a5b9611a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:32.480543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a812c-fe32-4e97-bf04-a88a5b9611a6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:32.480828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a812c-fe32-4e97-bf04-a88a5b9611a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:32.481104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a812c-fe32-4e97-bf04-a88a5b9611a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:32.492028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a812c-fe32-4e97-bf04-a88a5b9611a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-a94a812c-fe32-4e97-bf04-a88a5b9611a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:32.499603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:32.500726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:32.501976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 26/49] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:39:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:32.507151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976a96e-4e88-4b69-bd66-5c1c243675a4 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:32.581032 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:32.643652 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976a96e-4e88-4b69-bd66-5c1c243675a4 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:32.679623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3c424-454c-4d7d-b82b-eb7b9c2e9a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:32.680303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3c424-454c-4d7d-b82b-eb7b9c2e9a1f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:32.680618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3c424-454c-4d7d-b82b-eb7b9c2e9a1f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:32.680896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3c424-454c-4d7d-b82b-eb7b9c2e9a1f None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:32.681899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3c424-454c-4d7d-b82b-eb7b9c2e9a1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:32.682208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3c424-454c-4d7d-b82b-eb7b9c2e9a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:32.683588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3c424-454c-4d7d-b82b-eb7b9c2e9a1f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:32.696391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 24/50] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:32.702073 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a812c-fe32-4e97-bf04-a88a5b9611a6 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:32.768696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf3a7b-c6ee-4cc8-acb5-d9e0b30ec2b1 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:32.768905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf3a7b-c6ee-4cc8-acb5-d9e0b30ec2b1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:32.769211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf3a7b-c6ee-4cc8-acb5-d9e0b30ec2b1 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:32.770629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 25/51] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:39:32.780002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e9bb6-6c3b-4dc3-8ea2-7780df3e1022 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:32.780478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e9bb6-6c3b-4dc3-8ea2-7780df3e1022 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:32.780897 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e9bb6-6c3b-4dc3-8ea2-7780df3e1022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:32.799117 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-179e9bb6-6c3b-4dc3-8ea2-7780df3e1022 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:32.800377 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-179e9bb6-6c3b-4dc3-8ea2-7780df3e1022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:32.823217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a812c-fe32-4e97-bf04-a88a5b9611a6 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:32.859309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1c8c4-2031-46ef-bd41-a7f3233801b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:32.860185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1c8c4-2031-46ef-bd41-a7f3233801b9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:32.860510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1c8c4-2031-46ef-bd41-a7f3233801b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:32.860785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1c8c4-2031-46ef-bd41-a7f3233801b9 None admin] PATH_INFO: `/v3/users/adee0d7d3c614774bcfa96ea7aed77e6` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:32.861998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1c8c4-2031-46ef-bd41-a7f3233801b9 None admin] RBAC: Authorizing `identity:get_user(user_id=adee0d7d3c614774bcfa96ea7aed77e6)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:32.863440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1c8c4-2031-46ef-bd41-a7f3233801b9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:32.865112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 27/52] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:32 2022] GET /identity/v3/users/adee0d7d3c614774bcfa96ea7aed77e6 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:32.870769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1c8c4-2031-46ef-bd41-a7f3233801b9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:32.871674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 26/53] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:32.896009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e9bb6-6c3b-4dc3-8ea2-7780df3e1022 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:32.904690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:32.944665 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:32.984353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1c8c4-2031-46ef-bd41-a7f3233801b9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:33.005072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e9bb6-6c3b-4dc3-8ea2-7780df3e1022 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:33.028253 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d0c75-332d-4291-843c-4337aef7e75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:33.029719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d0c75-332d-4291-843c-4337aef7e75b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:33.030189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d0c75-332d-4291-843c-4337aef7e75b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:33.030777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d0c75-332d-4291-843c-4337aef7e75b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:33.032847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d0c75-332d-4291-843c-4337aef7e75b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:33.035502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d0c75-332d-4291-843c-4337aef7e75b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:33.038866 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 28/54] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:33.044854 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d0c75-332d-4291-843c-4337aef7e75b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:33.052708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d40b8-2e52-4d6d-9ffa-26c4d4e0e15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:33.054009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d40b8-2e52-4d6d-9ffa-26c4d4e0e15b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:33.054330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d40b8-2e52-4d6d-9ffa-26c4d4e0e15b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:33.054618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d40b8-2e52-4d6d-9ffa-26c4d4e0e15b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:33.056541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d40b8-2e52-4d6d-9ffa-26c4d4e0e15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:33.058494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d40b8-2e52-4d6d-9ffa-26c4d4e0e15b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:33.060299 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 27/55] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:33.066046 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d40b8-2e52-4d6d-9ffa-26c4d4e0e15b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:33.175657 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d40b8-2e52-4d6d-9ffa-26c4d4e0e15b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:33.185969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:33.186292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:33.189983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d0c75-332d-4291-843c-4337aef7e75b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:33.216762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f6b1e-5845-4842-93c8-b5803ef8f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:33.217474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f6b1e-5845-4842-93c8-b5803ef8f706 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:33.217803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f6b1e-5845-4842-93c8-b5803ef8f706 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:33.218123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f6b1e-5845-4842-93c8-b5803ef8f706 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:33.218881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f6b1e-5845-4842-93c8-b5803ef8f706 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:33.220531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f6b1e-5845-4842-93c8-b5803ef8f706 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:33.226747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def9303-90b1-4903-8499-accd5097907c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:33.227412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def9303-90b1-4903-8499-accd5097907c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:33.227702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def9303-90b1-4903-8499-accd5097907c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:33.227983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def9303-90b1-4903-8499-accd5097907c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:33.228855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def9303-90b1-4903-8499-accd5097907c None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.domain.id=default, user.id=adee0d7d3c614774bcfa96ea7aed77e6) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:33.229215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def9303-90b1-4903-8499-accd5097907c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:33.230655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def9303-90b1-4903-8499-accd5097907c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:33.236499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 29/56] 10.222.0.85 () {56 vars in 1322 bytes} [Tue Jul 19 20:39:33 2022] GET /identity/v3/role_assignments?user.id=adee0d7d3c614774bcfa96ea7aed77e6&scope.domain.id=default&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 530 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:33.251474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 28/57] 10.222.0.85 () {60 vars in 1140 bytes} [Tue Jul 19 20:39:33 2022] POST /identity/v3/projects => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:39:35.310157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4bff7-e65a-4d84-be10-46a8d062ec0e None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:35.310686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4bff7-e65a-4d84-be10-46a8d062ec0e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:35.310770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4bff7-e65a-4d84-be10-46a8d062ec0e None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:35.312099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 30/58] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:35.317207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46b55f-9b31-4916-856f-5322477d0883 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:35.317605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46b55f-9b31-4916-856f-5322477d0883 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:35.317770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46b55f-9b31-4916-856f-5322477d0883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:35.334747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-fe46b55f-9b31-4916-856f-5322477d0883 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:35.336549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fe46b55f-9b31-4916-856f-5322477d0883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:35.404732 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 29/59] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:35.429868 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4bff7-e65a-4d84-be10-46a8d062ec0e None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:35.544189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4bff7-e65a-4d84-be10-46a8d062ec0e None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:35.585505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3707525-379e-424d-a207-9ac290d58fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:35.586304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3707525-379e-424d-a207-9ac290d58fa7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:35.586592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3707525-379e-424d-a207-9ac290d58fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:35.586872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3707525-379e-424d-a207-9ac290d58fa7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:35.588106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3707525-379e-424d-a207-9ac290d58fa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:35.589612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3707525-379e-424d-a207-9ac290d58fa7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:35.591254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 31/60] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:35.597059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46b55f-9b31-4916-856f-5322477d0883 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:35.701369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:35.713468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46b55f-9b31-4916-856f-5322477d0883 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:35.753175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8bcf2-b4fa-4b8e-8157-39058e553c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:35.754331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8bcf2-b4fa-4b8e-8157-39058e553c56 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:35.754844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8bcf2-b4fa-4b8e-8157-39058e553c56 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:35.755351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8bcf2-b4fa-4b8e-8157-39058e553c56 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:35.757268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8bcf2-b4fa-4b8e-8157-39058e553c56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:35.759979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8bcf2-b4fa-4b8e-8157-39058e553c56 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:35.786226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 30/61] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:39:35 2022] POST /identity/v3/users => generated 275 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:39:38.873925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a08bfe-4578-4451-8fa0-74c38c76772e None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:38.875220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a08bfe-4578-4451-8fa0-74c38c76772e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:38.876041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a08bfe-4578-4451-8fa0-74c38c76772e None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:38.878870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 32/62] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:38.885789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23fdb5-c7fb-4541-a234-d76ac3cda189 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:38.887309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23fdb5-c7fb-4541-a234-d76ac3cda189 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:38.888069 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23fdb5-c7fb-4541-a234-d76ac3cda189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:38.908608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ae23fdb5-c7fb-4541-a234-d76ac3cda189 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:38.910311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ae23fdb5-c7fb-4541-a234-d76ac3cda189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:38.968346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4bcc7-2784-40ee-82b2-f28be7dcafc1 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:38.968999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4bcc7-2784-40ee-82b2-f28be7dcafc1 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:38.969495 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4bcc7-2784-40ee-82b2-f28be7dcafc1 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:38.971225 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 33/63] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:39:38.977709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f23882-e0e6-40dd-9b0d-47f7e24a43dc None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:38.978299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f23882-e0e6-40dd-9b0d-47f7e24a43dc None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:38.978805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f23882-e0e6-40dd-9b0d-47f7e24a43dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:39.013500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-40f23882-e0e6-40dd-9b0d-47f7e24a43dc None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:39.014722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-40f23882-e0e6-40dd-9b0d-47f7e24a43dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:39.032965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 31/64] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:39.064515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23fdb5-c7fb-4541-a234-d76ac3cda189 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:39.088867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:39.145375 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 34/65] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:39.198027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f23882-e0e6-40dd-9b0d-47f7e24a43dc None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:39.215859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:39.243622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23fdb5-c7fb-4541-a234-d76ac3cda189 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:39.280684 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5849f-a26b-4ec7-8684-72a3ec80f84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:39.281739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5849f-a26b-4ec7-8684-72a3ec80f84f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:39.282175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5849f-a26b-4ec7-8684-72a3ec80f84f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:39.282600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5849f-a26b-4ec7-8684-72a3ec80f84f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:39.296888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5849f-a26b-4ec7-8684-72a3ec80f84f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-0ac5849f-a26b-4ec7-8684-72a3ec80f84f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:39.307904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:39.309297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:39.310350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:39:39.312535 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 32/66] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:39:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:39:39.319526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5849f-a26b-4ec7-8684-72a3ec80f84f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:39.422619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f23882-e0e6-40dd-9b0d-47f7e24a43dc None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:39.468806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68a026-b32b-4869-bd6a-c74640b2e00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:39.470038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68a026-b32b-4869-bd6a-c74640b2e00f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:39.470473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68a026-b32b-4869-bd6a-c74640b2e00f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:39.470891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68a026-b32b-4869-bd6a-c74640b2e00f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:39.486131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68a026-b32b-4869-bd6a-c74640b2e00f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:39.486477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5849f-a26b-4ec7-8684-72a3ec80f84f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-9f68a026-b32b-4869-bd6a-c74640b2e00f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:39.496447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:39.497565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:39.498549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:39.500796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 35/67] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:39:39.502483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209a34e-6d06-4cb0-a8cc-4f99acb87efc None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:39.502773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209a34e-6d06-4cb0-a8cc-4f99acb87efc None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:39.503060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209a34e-6d06-4cb0-a8cc-4f99acb87efc None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:39.504714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 36/68] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:39.510149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209a34e-6d06-4cb0-a8cc-4f99acb87efc None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:39.534512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926254b-968a-41d4-88aa-e7819963c9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:39.535993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926254b-968a-41d4-88aa-e7819963c9fa None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:39.536542 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926254b-968a-41d4-88aa-e7819963c9fa None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:39.537082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926254b-968a-41d4-88aa-e7819963c9fa None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:39.538085 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926254b-968a-41d4-88aa-e7819963c9fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:39.538798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926254b-968a-41d4-88aa-e7819963c9fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:39.541249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4926254b-968a-41d4-88aa-e7819963c9fa None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:39.543330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:39.546856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:39.558146 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 33/69] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:39.559957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29167c7-7dfc-4876-a606-efa5cb198c09 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:39.560493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29167c7-7dfc-4876-a606-efa5cb198c09 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:39.561287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29167c7-7dfc-4876-a606-efa5cb198c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:39.583442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c29167c7-7dfc-4876-a606-efa5cb198c09 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:39.586263 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c29167c7-7dfc-4876-a606-efa5cb198c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:39.656658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209a34e-6d06-4cb0-a8cc-4f99acb87efc None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:39.692470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 34/70] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:39.692470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29167c7-7dfc-4876-a606-efa5cb198c09 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:39.718289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00dfb5-504a-46b7-8462-5ef37c4c0ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:39.719455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00dfb5-504a-46b7-8462-5ef37c4c0ad7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:39.720026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00dfb5-504a-46b7-8462-5ef37c4c0ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:39.720604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00dfb5-504a-46b7-8462-5ef37c4c0ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:39.721838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00dfb5-504a-46b7-8462-5ef37c4c0ad7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:39.722438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00dfb5-504a-46b7-8462-5ef37c4c0ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:39.724980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00dfb5-504a-46b7-8462-5ef37c4c0ad7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:39.739374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 37/71] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:39:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:39.741293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fed17-db95-4605-82c6-7d10ca01460a None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:39.741844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fed17-db95-4605-82c6-7d10ca01460a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:39.742390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fed17-db95-4605-82c6-7d10ca01460a None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:39.750789 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 38/72] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:39.750789 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5fed17-db95-4605-82c6-7d10ca01460a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:39.828630 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29167c7-7dfc-4876-a606-efa5cb198c09 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:39.847050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:39.860083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:39.867780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5fed17-db95-4605-82c6-7d10ca01460a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:39.868782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c6f21-07c4-4326-a4bc-a331933e7a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:39.869720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c6f21-07c4-4326-a4bc-a331933e7a1f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:39.870777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c6f21-07c4-4326-a4bc-a331933e7a1f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:39.870777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c6f21-07c4-4326-a4bc-a331933e7a1f None admin] PATH_INFO: `/v3/users/adee0d7d3c614774bcfa96ea7aed77e6` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:39.871747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c6f21-07c4-4326-a4bc-a331933e7a1f None admin] RBAC: Authorizing `identity:get_user(user_id=adee0d7d3c614774bcfa96ea7aed77e6)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:39.873238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c6f21-07c4-4326-a4bc-a331933e7a1f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:39.875094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 35/73] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:39 2022] GET /identity/v3/users/adee0d7d3c614774bcfa96ea7aed77e6 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:39.876015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c6f21-07c4-4326-a4bc-a331933e7a1f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:39.905555 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd950050-9719-4085-8b56-dc65da18e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:39.906357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950050-9719-4085-8b56-dc65da18e0ec None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:39.906643 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950050-9719-4085-8b56-dc65da18e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:39.906923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950050-9719-4085-8b56-dc65da18e0ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:39.923235 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd950050-9719-4085-8b56-dc65da18e0ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-bd950050-9719-4085-8b56-dc65da18e0ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:39.939270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:39.940196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:39.941012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:39.941601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 39/74] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:39.944672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf452407-b0d4-40d0-bd28-df3301422741 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:39.945127 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf452407-b0d4-40d0-bd28-df3301422741 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:39.945623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf452407-b0d4-40d0-bd28-df3301422741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:39.972539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-cf452407-b0d4-40d0-bd28-df3301422741 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:39.974487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cf452407-b0d4-40d0-bd28-df3301422741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:39.991081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c6f21-07c4-4326-a4bc-a331933e7a1f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:40.031250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f5f3b-4489-4f11-ac7a-fb182604f42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:40.032134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f5f3b-4489-4f11-ac7a-fb182604f42d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:40.032460 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f5f3b-4489-4f11-ac7a-fb182604f42d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:40.032745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f5f3b-4489-4f11-ac7a-fb182604f42d None admin] PATH_INFO: `/v3/users/7aee709fda854364a32f39ff15c6aafa` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:40.049193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f5f3b-4489-4f11-ac7a-fb182604f42d None admin] RBAC: Authorizing `identity:get_user(user_id=7aee709fda854364a32f39ff15c6aafa)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:40.050674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f5f3b-4489-4f11-ac7a-fb182604f42d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:40.052692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 36/75] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:39 2022] GET /identity/v3/users/7aee709fda854364a32f39ff15c6aafa => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:40.053779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f5f3b-4489-4f11-ac7a-fb182604f42d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:40.054229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 40/76] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:40.055708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a37ffd-db44-4b82-b25f-b263fb04d30a None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:40.056511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a37ffd-db44-4b82-b25f-b263fb04d30a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:40.056511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a37ffd-db44-4b82-b25f-b263fb04d30a None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:40.058666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 41/77] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:40.059723 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a37ffd-db44-4b82-b25f-b263fb04d30a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:40.130643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:40.149372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:40.175297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a37ffd-db44-4b82-b25f-b263fb04d30a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:40.181320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f5f3b-4489-4f11-ac7a-fb182604f42d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:40.212424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc50fb-7f2a-4184-8264-1d74dd3ad0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:40.213039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc50fb-7f2a-4184-8264-1d74dd3ad0b9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:40.213341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc50fb-7f2a-4184-8264-1d74dd3ad0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:40.213616 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc50fb-7f2a-4184-8264-1d74dd3ad0b9 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:40.214369 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc50fb-7f2a-4184-8264-1d74dd3ad0b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:40.214685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc50fb-7f2a-4184-8264-1d74dd3ad0b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:40.216097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc50fb-7f2a-4184-8264-1d74dd3ad0b9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:40.220761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6079bf-359b-4eb1-9b0b-e21ef9fbe29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:40.221421 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6079bf-359b-4eb1-9b0b-e21ef9fbe29e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:40.221708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6079bf-359b-4eb1-9b0b-e21ef9fbe29e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:40.221983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6079bf-359b-4eb1-9b0b-e21ef9fbe29e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:40.228385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 42/78] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:39:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:40.228385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc50fb-7f2a-4184-8264-1d74dd3ad0b9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:40.292842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6079bf-359b-4eb1-9b0b-e21ef9fbe29e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:40.294320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6079bf-359b-4eb1-9b0b-e21ef9fbe29e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:40.326372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc50fb-7f2a-4184-8264-1d74dd3ad0b9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:40.348785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:40.363322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab582a15-03aa-4392-886e-c95aa23df25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:40.363976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab582a15-03aa-4392-886e-c95aa23df25e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:40.364262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab582a15-03aa-4392-886e-c95aa23df25e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:40.364573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab582a15-03aa-4392-886e-c95aa23df25e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:40.368915 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:40.397418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 37/79] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:39:40.398854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bce98-c2d5-468e-993f-5892a3991aff None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:40.399255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bce98-c2d5-468e-993f-5892a3991aff None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:40.399491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bce98-c2d5-468e-993f-5892a3991aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:40.419084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e47bce98-c2d5-468e-993f-5892a3991aff None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:40.420351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e47bce98-c2d5-468e-993f-5892a3991aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:40.437421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab582a15-03aa-4392-886e-c95aa23df25e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:40.438891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab582a15-03aa-4392-886e-c95aa23df25e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:40.502419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 38/80] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:40.502419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bce98-c2d5-468e-993f-5892a3991aff None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:40.555334 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 43/81] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:39:40.557171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab582a15-03aa-4392-886e-c95aa23df25e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:40.593001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:40.601292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:40.618507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bce98-c2d5-468e-993f-5892a3991aff None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:40.654343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867bdf8-1daa-45b9-ba33-80a09c7cbfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:40.655155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867bdf8-1daa-45b9-ba33-80a09c7cbfb2 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:40.655450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867bdf8-1daa-45b9-ba33-80a09c7cbfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:40.655731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867bdf8-1daa-45b9-ba33-80a09c7cbfb2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:40.659552 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab582a15-03aa-4392-886e-c95aa23df25e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:40.665798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867bdf8-1daa-45b9-ba33-80a09c7cbfb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-e867bdf8-1daa-45b9-ba33-80a09c7cbfb2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:40.673933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:40.675050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:39:40.675846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 39/82] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:39:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:40.676514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867bdf8-1daa-45b9-ba33-80a09c7cbfb2 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:40.699425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0c554-cdf9-4cf7-b7e5-3793df5e3277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:40.700297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0c554-cdf9-4cf7-b7e5-3793df5e3277 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:40.700615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0c554-cdf9-4cf7-b7e5-3793df5e3277 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:40.700894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0c554-cdf9-4cf7-b7e5-3793df5e3277 None admin] PATH_INFO: `/v3/users/7aee709fda854364a32f39ff15c6aafa` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:40.702120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0c554-cdf9-4cf7-b7e5-3793df5e3277 None admin] RBAC: Authorizing `identity:get_user(user_id=7aee709fda854364a32f39ff15c6aafa)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:40.703599 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0c554-cdf9-4cf7-b7e5-3793df5e3277 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:40.705412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 44/83] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:40 2022] GET /identity/v3/users/7aee709fda854364a32f39ff15c6aafa => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:40.706182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0c554-cdf9-4cf7-b7e5-3793df5e3277 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:40.784353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867bdf8-1daa-45b9-ba33-80a09c7cbfb2 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:40.810069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0c554-cdf9-4cf7-b7e5-3793df5e3277 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:40.812946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:40.830292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fbf00-727a-4af9-a428-0a2cf1e3f57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:40.833706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fbf00-727a-4af9-a428-0a2cf1e3f57b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:40.833706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fbf00-727a-4af9-a428-0a2cf1e3f57b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:40.833706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fbf00-727a-4af9-a428-0a2cf1e3f57b None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:40.834514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fbf00-727a-4af9-a428-0a2cf1e3f57b None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:40.836707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fbf00-727a-4af9-a428-0a2cf1e3f57b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:40.839051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 40/84] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:40 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:40.839601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fbf00-727a-4af9-a428-0a2cf1e3f57b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:40.856023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:40.856686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306edd6e-6613-49ba-9a46-f321684f8064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:40.857261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306edd6e-6613-49ba-9a46-f321684f8064 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:40.857572 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306edd6e-6613-49ba-9a46-f321684f8064 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:40.858017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306edd6e-6613-49ba-9a46-f321684f8064 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:40.859289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306edd6e-6613-49ba-9a46-f321684f8064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:40.860792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306edd6e-6613-49ba-9a46-f321684f8064 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:40.862439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 45/85] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:40.868010 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306edd6e-6613-49ba-9a46-f321684f8064 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:40.964196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fbf00-727a-4af9-a428-0a2cf1e3f57b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:40.978737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306edd6e-6613-49ba-9a46-f321684f8064 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:41.003596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbaf08-5939-4c8f-8afd-337beeb34337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:41.004507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbaf08-5939-4c8f-8afd-337beeb34337 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:41.004800 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbaf08-5939-4c8f-8afd-337beeb34337 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:41.005083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbaf08-5939-4c8f-8afd-337beeb34337 None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:41.006526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cbaf08-5939-4c8f-8afd-337beeb34337 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:41.008568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cbaf08-5939-4c8f-8afd-337beeb34337 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:41.010307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 41/86] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:40 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:41.011091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbaf08-5939-4c8f-8afd-337beeb34337 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:41.022155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d3219-e4f2-4437-9a8f-34af6f74a32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:41.022834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d3219-e4f2-4437-9a8f-34af6f74a32a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:41.023125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d3219-e4f2-4437-9a8f-34af6f74a32a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:41.023410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d3219-e4f2-4437-9a8f-34af6f74a32a None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:41.024169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d3219-e4f2-4437-9a8f-34af6f74a32a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:41.024515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d3219-e4f2-4437-9a8f-34af6f74a32a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:41.025902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d3219-e4f2-4437-9a8f-34af6f74a32a None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:41.034441 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 46/87] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:39:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:41.035174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d3219-e4f2-4437-9a8f-34af6f74a32a None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:41.072505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:41.096301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:41.116981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbaf08-5939-4c8f-8afd-337beeb34337 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:41.150604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d3219-e4f2-4437-9a8f-34af6f74a32a None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:41.154371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca699-5329-42e6-aacb-c8c9d1d79faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:41.155032 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca699-5329-42e6-aacb-c8c9d1d79faa None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:41.155325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca699-5329-42e6-aacb-c8c9d1d79faa None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:41.155612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca699-5329-42e6-aacb-c8c9d1d79faa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:41.186327 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811cc7d-1041-4904-8057-f07a507d3b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:41.186988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811cc7d-1041-4904-8057-f07a507d3b3e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:41.187272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811cc7d-1041-4904-8057-f07a507d3b3e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:41.187548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811cc7d-1041-4904-8057-f07a507d3b3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:41.188396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811cc7d-1041-4904-8057-f07a507d3b3e None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=e1b269e3c66347e1aa9595854069b963, user.id=adee0d7d3c614774bcfa96ea7aed77e6) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:41.188709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811cc7d-1041-4904-8057-f07a507d3b3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:41.190162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811cc7d-1041-4904-8057-f07a507d3b3e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:41.197124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 47/88] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:41 2022] GET /identity/v3/role_assignments?user.id=adee0d7d3c614774bcfa96ea7aed77e6&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:41.198434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811cc7d-1041-4904-8057-f07a507d3b3e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:41.264184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aca699-5329-42e6-aacb-c8c9d1d79faa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:41.266510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aca699-5329-42e6-aacb-c8c9d1d79faa None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:41.366161 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:41.371638 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811cc7d-1041-4904-8057-f07a507d3b3e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:41.401082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:41.433373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 42/89] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:39:41.433597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c969ca-612c-48d7-ac93-653d18ad4bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:41.434527 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c969ca-612c-48d7-ac93-653d18ad4bbe None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:41.435025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c969ca-612c-48d7-ac93-653d18ad4bbe None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:41.435526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c969ca-612c-48d7-ac93-653d18ad4bbe None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:41.435768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca699-5329-42e6-aacb-c8c9d1d79faa None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:41.436928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c969ca-612c-48d7-ac93-653d18ad4bbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:41.439705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c969ca-612c-48d7-ac93-653d18ad4bbe None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:41.481642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 48/90] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:39:41 2022] POST /identity/v3/users => generated 289 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:39:41.482922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c969ca-612c-48d7-ac93-653d18ad4bbe None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:41.583718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca699-5329-42e6-aacb-c8c9d1d79faa None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:41.628244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50cc27-53fc-4706-aae1-39903465d0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:41.629023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50cc27-53fc-4706-aae1-39903465d0a8 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:41.629388 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50cc27-53fc-4706-aae1-39903465d0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:41.629741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50cc27-53fc-4706-aae1-39903465d0a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:41.630770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50cc27-53fc-4706-aae1-39903465d0a8 None admin] RBAC: Adding query filter params (role.id=0f8e73b285204863a74f8af4ff7bbc86, scope.project.id=e1b269e3c66347e1aa9595854069b963, user.id=7aee709fda854364a32f39ff15c6aafa) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:41.631302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50cc27-53fc-4706-aae1-39903465d0a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:41.633824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50cc27-53fc-4706-aae1-39903465d0a8 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:41.646430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 43/91] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:41 2022] GET /identity/v3/role_assignments?user.id=7aee709fda854364a32f39ff15c6aafa&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:41.647454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50cc27-53fc-4706-aae1-39903465d0a8 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:41.648409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c969ca-612c-48d7-ac93-653d18ad4bbe None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:41.709794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03010a31-2e64-4243-a550-d6bc3cc90b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:41.711338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03010a31-2e64-4243-a550-d6bc3cc90b94 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:41.711870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03010a31-2e64-4243-a550-d6bc3cc90b94 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:41.712372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03010a31-2e64-4243-a550-d6bc3cc90b94 None admin] PATH_INFO: `/v3/users/7aee709fda854364a32f39ff15c6aafa` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:41.713774 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:41.714384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03010a31-2e64-4243-a550-d6bc3cc90b94 None admin] RBAC: Authorizing `identity:get_user(user_id=7aee709fda854364a32f39ff15c6aafa)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:41.717080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03010a31-2e64-4243-a550-d6bc3cc90b94 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:41.719880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 49/92] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:41 2022] GET /identity/v3/users/7aee709fda854364a32f39ff15c6aafa => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:41.725898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03010a31-2e64-4243-a550-d6bc3cc90b94 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:41.757785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:41.791891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50cc27-53fc-4706-aae1-39903465d0a8 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:41.839542 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f93eb8-8aa5-4d51-b094-140650dd54a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:41.840832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f93eb8-8aa5-4d51-b094-140650dd54a9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:41.841320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f93eb8-8aa5-4d51-b094-140650dd54a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:41.841978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f93eb8-8aa5-4d51-b094-140650dd54a9 None admin] PATH_INFO: `/v3/projects/7fc29e50692c4f1a866578fee9aa0044` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:41.843509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f93eb8-8aa5-4d51-b094-140650dd54a9 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc29e50692c4f1a866578fee9aa0044)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:41.845609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f93eb8-8aa5-4d51-b094-140650dd54a9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:41.847937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 44/93] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:41 2022] GET /identity/v3/projects/7fc29e50692c4f1a866578fee9aa0044 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:41.855291 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f93eb8-8aa5-4d51-b094-140650dd54a9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:41.878513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03010a31-2e64-4243-a550-d6bc3cc90b94 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:41.927970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193603f6-066a-4126-8524-f7ffdac207e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:41.929639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193603f6-066a-4126-8524-f7ffdac207e7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:41.930324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193603f6-066a-4126-8524-f7ffdac207e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:41.930755 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193603f6-066a-4126-8524-f7ffdac207e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:41.981934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f93eb8-8aa5-4d51-b094-140650dd54a9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:41.999061 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:42.026846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d7edc-9c47-45d8-9251-d3ecfd8bd349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:42.028417 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d7edc-9c47-45d8-9251-d3ecfd8bd349 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:42.028968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d7edc-9c47-45d8-9251-d3ecfd8bd349 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:42.029447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d7edc-9c47-45d8-9251-d3ecfd8bd349 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:42.030483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d7edc-9c47-45d8-9251-d3ecfd8bd349 None admin] RBAC: Adding query filter params (role.id=0f8e73b285204863a74f8af4ff7bbc86, scope.project.id=7fc29e50692c4f1a866578fee9aa0044, user.id=7aee709fda854364a32f39ff15c6aafa) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:42.030996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d7edc-9c47-45d8-9251-d3ecfd8bd349 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:42.032683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d7edc-9c47-45d8-9251-d3ecfd8bd349 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:42.039742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 45/94] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:41 2022] GET /identity/v3/role_assignments?user.id=7aee709fda854364a32f39ff15c6aafa&scope.project.id=7fc29e50692c4f1a866578fee9aa0044&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:42.044957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193603f6-066a-4126-8524-f7ffdac207e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:42.047482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193603f6-066a-4126-8524-f7ffdac207e7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:42.093906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:42.228044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 50/95] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:39:42.236021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d7edc-9c47-45d8-9251-d3ecfd8bd349 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:42.353881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d7edc-9c47-45d8-9251-d3ecfd8bd349 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:42.395582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f728a6a-d488-4bbe-8f01-120c4f2e728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:42.396502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728a6a-d488-4bbe-8f01-120c4f2e728c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:42.397346 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728a6a-d488-4bbe-8f01-120c4f2e728c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:42.397630 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f728a6a-d488-4bbe-8f01-120c4f2e728c None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:42.398902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f728a6a-d488-4bbe-8f01-120c4f2e728c None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:42.400504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f728a6a-d488-4bbe-8f01-120c4f2e728c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:42.402467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 46/96] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:42 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:42.408978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193603f6-066a-4126-8524-f7ffdac207e7 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:42.545418 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193603f6-066a-4126-8524-f7ffdac207e7 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:42.572936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:42.596136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f03d8f-6d7d-4da7-9b83-acd0dfd66a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:42.596843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f03d8f-6d7d-4da7-9b83-acd0dfd66a9b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:42.597136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f03d8f-6d7d-4da7-9b83-acd0dfd66a9b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:42.597420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f03d8f-6d7d-4da7-9b83-acd0dfd66a9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:42.598275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f03d8f-6d7d-4da7-9b83-acd0dfd66a9b None admin] RBAC: Adding query filter params (role.id=aab847245001401cb4d3d43238298f07, scope.project.id=e1b269e3c66347e1aa9595854069b963, user.id=7aee709fda854364a32f39ff15c6aafa) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:42.598592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f03d8f-6d7d-4da7-9b83-acd0dfd66a9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:42.600124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f03d8f-6d7d-4da7-9b83-acd0dfd66a9b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:42.607260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 51/97] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:42 2022] GET /identity/v3/role_assignments?user.id=7aee709fda854364a32f39ff15c6aafa&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=aab847245001401cb4d3d43238298f07 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:43.885062 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8526af-5645-4122-b6c3-3ac5f1a1fa46 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:43.885588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8526af-5645-4122-b6c3-3ac5f1a1fa46 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:43.885660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8526af-5645-4122-b6c3-3ac5f1a1fa46 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:43.887553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 47/98] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:43.893116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61148e2-cee2-4867-a097-a333719252ee None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:43.893612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61148e2-cee2-4867-a097-a333719252ee None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:43.893682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61148e2-cee2-4867-a097-a333719252ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:43.913311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-a61148e2-cee2-4867-a097-a333719252ee None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:43.914547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a61148e2-cee2-4867-a097-a333719252ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:44.021397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 52/99] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:39:44.052591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8526af-5645-4122-b6c3-3ac5f1a1fa46 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:44.116707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:44.182896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8526af-5645-4122-b6c3-3ac5f1a1fa46 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:44.225603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f06c43-74b4-4ff4-a7d0-7aabe0b0c551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:44.226425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f06c43-74b4-4ff4-a7d0-7aabe0b0c551 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:44.226717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f06c43-74b4-4ff4-a7d0-7aabe0b0c551 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:44.227004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f06c43-74b4-4ff4-a7d0-7aabe0b0c551 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:44.242457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f06c43-74b4-4ff4-a7d0-7aabe0b0c551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-99f06c43-74b4-4ff4-a7d0-7aabe0b0c551 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:44.255436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:44.256922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:39:44.257893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 48/100] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:39:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:44.263216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61148e2-cee2-4867-a097-a333719252ee None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:44.417401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61148e2-cee2-4867-a097-a333719252ee None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:44.446863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:44.469638 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9083029-4acf-4568-893a-47f10ee32a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:44.470334 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9083029-4acf-4568-893a-47f10ee32a2a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:44.470625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9083029-4acf-4568-893a-47f10ee32a2a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:44.470906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9083029-4acf-4568-893a-47f10ee32a2a None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:44.471674 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9083029-4acf-4568-893a-47f10ee32a2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:44.471985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9083029-4acf-4568-893a-47f10ee32a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:44.473433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9083029-4acf-4568-893a-47f10ee32a2a None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:44.507523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 53/101] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:44.514748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f06c43-74b4-4ff4-a7d0-7aabe0b0c551 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:44.575112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b8ce3-2642-48c3-99a8-e788396e310d None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:44.575750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b8ce3-2642-48c3-99a8-e788396e310d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:44.576311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b8ce3-2642-48c3-99a8-e788396e310d None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:44.579024 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 54/102] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:44.601162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de2ac-9522-430f-85f2-bb0899bd4e15 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:44.601821 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de2ac-9522-430f-85f2-bb0899bd4e15 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:44.602301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de2ac-9522-430f-85f2-bb0899bd4e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:44.648451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-b93de2ac-9522-430f-85f2-bb0899bd4e15 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:44.650046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b93de2ac-9522-430f-85f2-bb0899bd4e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:44.777870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f06c43-74b4-4ff4-a7d0-7aabe0b0c551 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:44.781494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:44.885461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38358e-8a39-447d-9fec-6c75a01f8a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:44.886187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38358e-8a39-447d-9fec-6c75a01f8a04 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:44.886687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38358e-8a39-447d-9fec-6c75a01f8a04 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:44.886976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38358e-8a39-447d-9fec-6c75a01f8a04 None admin] PATH_INFO: `/v3/users/adee0d7d3c614774bcfa96ea7aed77e6` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:44.893175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38358e-8a39-447d-9fec-6c75a01f8a04 None admin] RBAC: Authorizing `identity:get_user(user_id=adee0d7d3c614774bcfa96ea7aed77e6)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:44.894900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38358e-8a39-447d-9fec-6c75a01f8a04 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:44.896936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 49/103] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:44 2022] GET /identity/v3/users/adee0d7d3c614774bcfa96ea7aed77e6 => generated 247 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:44.906389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 55/104] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:44.908167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38358e-8a39-447d-9fec-6c75a01f8a04 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:44.971148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93de2ac-9522-430f-85f2-bb0899bd4e15 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:45.245585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93de2ac-9522-430f-85f2-bb0899bd4e15 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:45.250483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38358e-8a39-447d-9fec-6c75a01f8a04 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:45.276354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:45.346299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c0d64-cb88-42d5-83b4-bd24c6ccba24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:45.347292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c0d64-cb88-42d5-83b4-bd24c6ccba24 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:45.347806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c0d64-cb88-42d5-83b4-bd24c6ccba24 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:45.348291 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c0d64-cb88-42d5-83b4-bd24c6ccba24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:45.369728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c0d64-cb88-42d5-83b4-bd24c6ccba24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-e59c0d64-cb88-42d5-83b4-bd24c6ccba24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:45.378559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:45.379537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:45.380363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:45.383210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59687371-d92d-4053-b40c-2f0275bb5737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:45.383622 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 56/105] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:45.384100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687371-d92d-4053-b40c-2f0275bb5737 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:45.384413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687371-d92d-4053-b40c-2f0275bb5737 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:45.384795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687371-d92d-4053-b40c-2f0275bb5737 None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:45.385939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637980b7-d2ee-4894-8bc3-c2567a4fd813 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:45.386439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637980b7-d2ee-4894-8bc3-c2567a4fd813 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:45.386913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637980b7-d2ee-4894-8bc3-c2567a4fd813 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:45.389213 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 57/106] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:45.390116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59687371-d92d-4053-b40c-2f0275bb5737 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:45.391180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637980b7-d2ee-4894-8bc3-c2567a4fd813 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:45.391623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59687371-d92d-4053-b40c-2f0275bb5737 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:45.393814 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 50/107] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:44 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:45.400010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889fb6-7ce0-43b0-bdce-75fa8586c289 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:45.400618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889fb6-7ce0-43b0-bdce-75fa8586c289 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:45.401053 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889fb6-7ce0-43b0-bdce-75fa8586c289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:45.439699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-89889fb6-7ce0-43b0-bdce-75fa8586c289 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:45.441418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-89889fb6-7ce0-43b0-bdce-75fa8586c289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:45.502268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:45.557271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637980b7-d2ee-4894-8bc3-c2567a4fd813 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:45.570664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 51/108] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:39:45.571881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89889fb6-7ce0-43b0-bdce-75fa8586c289 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:45.616069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca2482-cf1d-42de-a7dd-f6f7ebbe41e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:45.617399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca2482-cf1d-42de-a7dd-f6f7ebbe41e7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:45.617896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca2482-cf1d-42de-a7dd-f6f7ebbe41e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:45.618389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca2482-cf1d-42de-a7dd-f6f7ebbe41e7 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:45.620171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca2482-cf1d-42de-a7dd-f6f7ebbe41e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:45.620673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca2482-cf1d-42de-a7dd-f6f7ebbe41e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:45.623267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca2482-cf1d-42de-a7dd-f6f7ebbe41e7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:45.637202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 58/109] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:39:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:45.638894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a9963-6254-4c39-8205-d615c0b8ae09 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:45.639191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a9963-6254-4c39-8205-d615c0b8ae09 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:45.639474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a9963-6254-4c39-8205-d615c0b8ae09 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:45.640879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 59/110] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:45.641728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a9963-6254-4c39-8205-d615c0b8ae09 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:45.688774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:45.719381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89889fb6-7ce0-43b0-bdce-75fa8586c289 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:45.782686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a0110-d756-41a6-b2ad-72a860c7a2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:45.784078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a0110-d756-41a6-b2ad-72a860c7a2d9 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:45.784443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a0110-d756-41a6-b2ad-72a860c7a2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:45.784774 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a0110-d756-41a6-b2ad-72a860c7a2d9 None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963/users/adee0d7d3c614774bcfa96ea7aed77e6/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:45.789208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a0110-d756-41a6-b2ad-72a860c7a2d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1b269e3c66347e1aa9595854069b963, user_id=adee0d7d3c614774bcfa96ea7aed77e6, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:45.790946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a0110-d756-41a6-b2ad-72a860c7a2d9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:45.809974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 52/111] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:39:45 2022] PUT /identity/v3/projects/e1b269e3c66347e1aa9595854069b963/users/adee0d7d3c614774bcfa96ea7aed77e6/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:39:45.810660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a9963-6254-4c39-8205-d615c0b8ae09 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:45.811000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a0110-d756-41a6-b2ad-72a860c7a2d9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:45.849218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:45.865001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ade705-53b5-42a9-ac8c-1abc37740db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:45.867086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ade705-53b5-42a9-ac8c-1abc37740db9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:45.867653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ade705-53b5-42a9-ac8c-1abc37740db9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:45.868231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ade705-53b5-42a9-ac8c-1abc37740db9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:45.883523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ade705-53b5-42a9-ac8c-1abc37740db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-58ade705-53b5-42a9-ac8c-1abc37740db9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:45.893321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:45.894297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:39:45.895066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:45.899015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 60/112] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:45.901123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328c7ec-3071-4f3b-ba6e-b5044d08c3c4 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:45.901893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328c7ec-3071-4f3b-ba6e-b5044d08c3c4 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:45.902541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328c7ec-3071-4f3b-ba6e-b5044d08c3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:45.930423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4328c7ec-3071-4f3b-ba6e-b5044d08c3c4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:45.939628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4328c7ec-3071-4f3b-ba6e-b5044d08c3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:45.950773 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a0110-d756-41a6-b2ad-72a860c7a2d9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:46.015165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dda69a-d27b-4e38-8da1-7c658e6da1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:46.016847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dda69a-d27b-4e38-8da1-7c658e6da1f6 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:46.017364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dda69a-d27b-4e38-8da1-7c658e6da1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:46.017864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dda69a-d27b-4e38-8da1-7c658e6da1f6 None admin] PATH_INFO: `/v3/users/7aee709fda854364a32f39ff15c6aafa` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:46.025038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dda69a-d27b-4e38-8da1-7c658e6da1f6 None admin] RBAC: Authorizing `identity:get_user(user_id=7aee709fda854364a32f39ff15c6aafa)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:46.027821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dda69a-d27b-4e38-8da1-7c658e6da1f6 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:46.031125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 53/113] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:45 2022] GET /identity/v3/users/7aee709fda854364a32f39ff15c6aafa => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:46.034521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dda69a-d27b-4e38-8da1-7c658e6da1f6 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:46.126467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 61/114] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:46.132505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328c7ec-3071-4f3b-ba6e-b5044d08c3c4 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:46.194790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:46.294163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:46.302304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dda69a-d27b-4e38-8da1-7c658e6da1f6 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:46.321532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328c7ec-3071-4f3b-ba6e-b5044d08c3c4 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:46.360891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b857351-ddf0-42f1-9368-dfa848fa1561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:46.361771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b857351-ddf0-42f1-9368-dfa848fa1561 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:46.362060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b857351-ddf0-42f1-9368-dfa848fa1561 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:46.362456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b857351-ddf0-42f1-9368-dfa848fa1561 None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:46.363707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b857351-ddf0-42f1-9368-dfa848fa1561 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:46.365220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b857351-ddf0-42f1-9368-dfa848fa1561 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:46.368303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427cec6c-cc5a-4e25-b4bd-9e21971eef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:46.369643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cec6c-cc5a-4e25-b4bd-9e21971eef28 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:46.370468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cec6c-cc5a-4e25-b4bd-9e21971eef28 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:46.371043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cec6c-cc5a-4e25-b4bd-9e21971eef28 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:46.371994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 62/115] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:46 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:46.372626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cec6c-cc5a-4e25-b4bd-9e21971eef28 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:46.373325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cec6c-cc5a-4e25-b4bd-9e21971eef28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:46.373482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341438c-11e6-4f56-b4f7-a86a36c1cb72 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:46.373693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341438c-11e6-4f56-b4f7-a86a36c1cb72 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:46.373958 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341438c-11e6-4f56-b4f7-a86a36c1cb72 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:46.375547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 63/116] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:46.376285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341438c-11e6-4f56-b4f7-a86a36c1cb72 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:46.381526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cec6c-cc5a-4e25-b4bd-9e21971eef28 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:46.401823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 54/117] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:39:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:46.409670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427cec6c-cc5a-4e25-b4bd-9e21971eef28 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:46.481701 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:46.496286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341438c-11e6-4f56-b4f7-a86a36c1cb72 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:46.535623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427cec6c-cc5a-4e25-b4bd-9e21971eef28 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:46.535842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4555457-e6b7-4e2a-b75b-94f79875f2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:46.536630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4555457-e6b7-4e2a-b75b-94f79875f2ed None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:46.536981 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4555457-e6b7-4e2a-b75b-94f79875f2ed None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:46.537483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4555457-e6b7-4e2a-b75b-94f79875f2ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:46.539603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4555457-e6b7-4e2a-b75b-94f79875f2ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:46.541183 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4555457-e6b7-4e2a-b75b-94f79875f2ed None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:46.547601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 64/118] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:46.547601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7599528-dc0e-418b-b2a9-cab8b9b72a26 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:46.547601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7599528-dc0e-418b-b2a9-cab8b9b72a26 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:46.547601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7599528-dc0e-418b-b2a9-cab8b9b72a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:46.567015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e7599528-dc0e-418b-b2a9-cab8b9b72a26 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:46.568943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e7599528-dc0e-418b-b2a9-cab8b9b72a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:46.576910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee0036-c0cc-4c6a-bcee-ce06072415fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:46.578006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee0036-c0cc-4c6a-bcee-ce06072415fc None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:46.578296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee0036-c0cc-4c6a-bcee-ce06072415fc None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:46.578575 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee0036-c0cc-4c6a-bcee-ce06072415fc None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963/users/7aee709fda854364a32f39ff15c6aafa/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:46.580846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee0036-c0cc-4c6a-bcee-ce06072415fc None admin] RBAC: Authorizing `identity:create_grant(project_id=e1b269e3c66347e1aa9595854069b963, user_id=7aee709fda854364a32f39ff15c6aafa, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:46.583465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee0036-c0cc-4c6a-bcee-ce06072415fc None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:46.599596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 55/119] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:39:46 2022] PUT /identity/v3/projects/e1b269e3c66347e1aa9595854069b963/users/7aee709fda854364a32f39ff15c6aafa/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:39:46.600590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee0036-c0cc-4c6a-bcee-ce06072415fc None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:46.632854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:46.681408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 65/120] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:46.683381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77227f8-8347-4d36-95d5-9274ae869791 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:46.684664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77227f8-8347-4d36-95d5-9274ae869791 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:46.684977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77227f8-8347-4d36-95d5-9274ae869791 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:46.691667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 66/121] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:46.691667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77227f8-8347-4d36-95d5-9274ae869791 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:46.776933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee0036-c0cc-4c6a-bcee-ce06072415fc None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:46.809572 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:46.829957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfe567-4c5d-49ac-b886-d334283149e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:46.831230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfe567-4c5d-49ac-b886-d334283149e5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:46.831525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfe567-4c5d-49ac-b886-d334283149e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:46.831826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfe567-4c5d-49ac-b886-d334283149e5 None admin] PATH_INFO: `/v3/users/b3e729e34c514993aae340dcfaf4e1d8` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:46.832103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77227f8-8347-4d36-95d5-9274ae869791 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:46.855995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfe567-4c5d-49ac-b886-d334283149e5 None admin] RBAC: Authorizing `identity:get_user(user_id=b3e729e34c514993aae340dcfaf4e1d8)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:46.858219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfe567-4c5d-49ac-b886-d334283149e5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:46.861241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 56/122] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:46 2022] GET /identity/v3/users/b3e729e34c514993aae340dcfaf4e1d8 => generated 289 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:46.863212 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f509f5c-f0e4-4104-be7a-401528ba2d8e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:46.863505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f509f5c-f0e4-4104-be7a-401528ba2d8e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:46.864077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f509f5c-f0e4-4104-be7a-401528ba2d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:46.892312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0f509f5c-f0e4-4104-be7a-401528ba2d8e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:46.893598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0f509f5c-f0e4-4104-be7a-401528ba2d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:46.899227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eee094-309b-4f60-80c1-f73ac55bc9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:46.900476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eee094-309b-4f60-80c1-f73ac55bc9f7 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:46.901042 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eee094-309b-4f60-80c1-f73ac55bc9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:46.901534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eee094-309b-4f60-80c1-f73ac55bc9f7 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:46.903715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eee094-309b-4f60-80c1-f73ac55bc9f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:46.908061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eee094-309b-4f60-80c1-f73ac55bc9f7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:46.963163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 67/123] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:39:46 2022] POST /identity/v3/projects => generated 302 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:39:46.969488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eee094-309b-4f60-80c1-f73ac55bc9f7 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:46.980957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 57/124] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:46.981714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f509f5c-f0e4-4104-be7a-401528ba2d8e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:46.998348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:47.118443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eee094-309b-4f60-80c1-f73ac55bc9f7 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:47.155070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f509f5c-f0e4-4104-be7a-401528ba2d8e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:47.161477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898bab2-7d96-4129-a68e-896602cc3105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:47.162876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898bab2-7d96-4129-a68e-896602cc3105 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:47.163156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898bab2-7d96-4129-a68e-896602cc3105 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:47.163440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898bab2-7d96-4129-a68e-896602cc3105 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:47.185410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898bab2-7d96-4129-a68e-896602cc3105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-d898bab2-7d96-4129-a68e-896602cc3105 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:47.193940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:47.196152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 68/125] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:47.198109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898bab2-7d96-4129-a68e-896602cc3105 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:47.201207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:47.229745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf30d3-6e45-4f34-822a-bf1f58fcedc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:47.230411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf30d3-6e45-4f34-822a-bf1f58fcedc3 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:47.230703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf30d3-6e45-4f34-822a-bf1f58fcedc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:47.230990 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf30d3-6e45-4f34-822a-bf1f58fcedc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:47.313351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf30d3-6e45-4f34-822a-bf1f58fcedc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:47.314876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf30d3-6e45-4f34-822a-bf1f58fcedc3 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:47.316237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898bab2-7d96-4129-a68e-896602cc3105 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:47.327285 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:47.359546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331545c-2332-4cd8-bbe4-8f5bd39fef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:47.360403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331545c-2332-4cd8-bbe4-8f5bd39fef57 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:47.360766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331545c-2332-4cd8-bbe4-8f5bd39fef57 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:47.361119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331545c-2332-4cd8-bbe4-8f5bd39fef57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:47.376256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331545c-2332-4cd8-bbe4-8f5bd39fef57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-7331545c-2332-4cd8-bbe4-8f5bd39fef57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:47.385897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:47.388456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 69/126] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:39:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:47.390384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331545c-2332-4cd8-bbe4-8f5bd39fef57 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:47.480935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:47.525341 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 58/127] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:39:47.525907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf30d3-6e45-4f34-822a-bf1f58fcedc3 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:47.566494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331545c-2332-4cd8-bbe4-8f5bd39fef57 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:47.654773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcc0a9-7856-464d-b082-fb910628a4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:47.655419 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcc0a9-7856-464d-b082-fb910628a4e2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:47.655890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcc0a9-7856-464d-b082-fb910628a4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:47.656357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcc0a9-7856-464d-b082-fb910628a4e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:47.659824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcc0a9-7856-464d-b082-fb910628a4e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:47.660372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcc0a9-7856-464d-b082-fb910628a4e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:47.663155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcc0a9-7856-464d-b082-fb910628a4e2 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:47.691695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 70/128] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:39:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:47.692971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcc0a9-7856-464d-b082-fb910628a4e2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:47.770965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf30d3-6e45-4f34-822a-bf1f58fcedc3 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:47.789355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:47.850953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cadeaa-f9b7-4343-9cb5-410608d2b18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:47.852070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cadeaa-f9b7-4343-9cb5-410608d2b18c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:47.852428 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cadeaa-f9b7-4343-9cb5-410608d2b18c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:47.852715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cadeaa-f9b7-4343-9cb5-410608d2b18c None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:47.853484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cadeaa-f9b7-4343-9cb5-410608d2b18c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:47.853794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cadeaa-f9b7-4343-9cb5-410608d2b18c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:47.855202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cadeaa-f9b7-4343-9cb5-410608d2b18c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:47.878744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 59/129] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:39:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:47.879636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cadeaa-f9b7-4343-9cb5-410608d2b18c None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:47.907763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcc0a9-7856-464d-b082-fb910628a4e2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:47.944862 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:47.987781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74614689-30f4-4f70-85ea-fd0cd6edf988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:47.988852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74614689-30f4-4f70-85ea-fd0cd6edf988 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:47.989145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74614689-30f4-4f70-85ea-fd0cd6edf988 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:47.989455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74614689-30f4-4f70-85ea-fd0cd6edf988 None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:47.993203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74614689-30f4-4f70-85ea-fd0cd6edf988 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:47.995023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74614689-30f4-4f70-85ea-fd0cd6edf988 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:47.997144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 71/130] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:47 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:48.002991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74614689-30f4-4f70-85ea-fd0cd6edf988 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:48.099389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cadeaa-f9b7-4343-9cb5-410608d2b18c None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:48.137292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd86134-b39f-4d50-981c-7f68ae27a583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:48.138317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd86134-b39f-4d50-981c-7f68ae27a583 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:48.138611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd86134-b39f-4d50-981c-7f68ae27a583 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:48.138890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd86134-b39f-4d50-981c-7f68ae27a583 None admin] PATH_INFO: `/v3/users/7aee709fda854364a32f39ff15c6aafa` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:48.143864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd86134-b39f-4d50-981c-7f68ae27a583 None admin] RBAC: Authorizing `identity:get_user(user_id=7aee709fda854364a32f39ff15c6aafa)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:48.143864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd86134-b39f-4d50-981c-7f68ae27a583 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:48.143864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 60/131] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:47 2022] GET /identity/v3/users/7aee709fda854364a32f39ff15c6aafa => generated 275 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:48.144422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd86134-b39f-4d50-981c-7f68ae27a583 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:48.194180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:48.266548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd86134-b39f-4d50-981c-7f68ae27a583 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:48.278717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74614689-30f4-4f70-85ea-fd0cd6edf988 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:48.306058 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12991c6-6b41-4d54-99d2-f8aeadfaab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:48.306979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12991c6-6b41-4d54-99d2-f8aeadfaab9c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:48.307267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12991c6-6b41-4d54-99d2-f8aeadfaab9c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:48.307551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12991c6-6b41-4d54-99d2-f8aeadfaab9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:48.308616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12991c6-6b41-4d54-99d2-f8aeadfaab9c None admin] RBAC: Adding query filter params (role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2, scope.project.id=e1b269e3c66347e1aa9595854069b963, user.id=b3e729e34c514993aae340dcfaf4e1d8) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:48.308616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12991c6-6b41-4d54-99d2-f8aeadfaab9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:48.311645 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12991c6-6b41-4d54-99d2-f8aeadfaab9c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:48.318284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 61/132] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:48 2022] GET /identity/v3/role_assignments?user.id=b3e729e34c514993aae340dcfaf4e1d8&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:48.319495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12991c6-6b41-4d54-99d2-f8aeadfaab9c None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:48.397668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30ce50-4523-4558-ad84-20309231d9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:48.398573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30ce50-4523-4558-ad84-20309231d9d3 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:48.398861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30ce50-4523-4558-ad84-20309231d9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:48.399141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30ce50-4523-4558-ad84-20309231d9d3 None admin] PATH_INFO: `/v3/users/7aee709fda854364a32f39ff15c6aafa` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:48.400421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30ce50-4523-4558-ad84-20309231d9d3 None admin] RBAC: Authorizing `identity:get_user(user_id=7aee709fda854364a32f39ff15c6aafa)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:48.401915 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30ce50-4523-4558-ad84-20309231d9d3 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:48.403801 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 72/133] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:47 2022] GET /identity/v3/users/7aee709fda854364a32f39ff15c6aafa => generated 275 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:48.409339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30ce50-4523-4558-ad84-20309231d9d3 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:48.438557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12991c6-6b41-4d54-99d2-f8aeadfaab9c None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:48.440136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:48.481031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70083fd6-5fae-4178-9a98-98214c2feeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:48.481880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083fd6-5fae-4178-9a98-98214c2feeeb None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:48.482176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083fd6-5fae-4178-9a98-98214c2feeeb None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:48.482458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083fd6-5fae-4178-9a98-98214c2feeeb None admin] PATH_INFO: `/v3/projects/7fc29e50692c4f1a866578fee9aa0044` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:48.483696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70083fd6-5fae-4178-9a98-98214c2feeeb None admin] RBAC: Authorizing `identity:get_project(project_id=7fc29e50692c4f1a866578fee9aa0044)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:48.485524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70083fd6-5fae-4178-9a98-98214c2feeeb None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:48.487701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 62/134] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:48 2022] GET /identity/v3/projects/7fc29e50692c4f1a866578fee9aa0044 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:48.494699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70083fd6-5fae-4178-9a98-98214c2feeeb None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:48.564742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:48.624224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30ce50-4523-4558-ad84-20309231d9d3 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:48.667822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291dc49-acee-48cf-b840-f1f32de0696e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:48.668806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291dc49-acee-48cf-b840-f1f32de0696e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:48.669098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291dc49-acee-48cf-b840-f1f32de0696e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:48.669382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291dc49-acee-48cf-b840-f1f32de0696e None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:48.671021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291dc49-acee-48cf-b840-f1f32de0696e None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:48.673048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291dc49-acee-48cf-b840-f1f32de0696e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:48.674925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 73/135] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:48 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:48.678520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1849b53-2e7e-46a3-85cd-56d6fb2e5ffc None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:48.678520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1849b53-2e7e-46a3-85cd-56d6fb2e5ffc None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:48.678520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1849b53-2e7e-46a3-85cd-56d6fb2e5ffc None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:48.680137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 74/136] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:48.681810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1849b53-2e7e-46a3-85cd-56d6fb2e5ffc None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:48.692011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70083fd6-5fae-4178-9a98-98214c2feeeb None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:48.761507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb47230-67ef-4915-a943-2229b43e508e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:48.762937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb47230-67ef-4915-a943-2229b43e508e None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:48.763505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb47230-67ef-4915-a943-2229b43e508e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:48.764061 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb47230-67ef-4915-a943-2229b43e508e None admin] PATH_INFO: `/v3/projects/7fc29e50692c4f1a866578fee9aa0044/users/7aee709fda854364a32f39ff15c6aafa/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:48.769826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb47230-67ef-4915-a943-2229b43e508e None admin] RBAC: Authorizing `identity:create_grant(project_id=7fc29e50692c4f1a866578fee9aa0044, user_id=7aee709fda854364a32f39ff15c6aafa, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:48.771804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb47230-67ef-4915-a943-2229b43e508e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:48.797752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 63/137] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:39:48 2022] PUT /identity/v3/projects/7fc29e50692c4f1a866578fee9aa0044/users/7aee709fda854364a32f39ff15c6aafa/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:39:48.799655 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8e001-6982-4a15-b181-ea6449a17c86 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:48.800156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8e001-6982-4a15-b181-ea6449a17c86 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:48.800654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8e001-6982-4a15-b181-ea6449a17c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:48.810715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1849b53-2e7e-46a3-85cd-56d6fb2e5ffc None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:48.836800 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-aed8e001-6982-4a15-b181-ea6449a17c86 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:48.838607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-aed8e001-6982-4a15-b181-ea6449a17c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:48.855010 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:48.889249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b026a-2dba-424c-820d-8112000839ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:48.890292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b026a-2dba-424c-820d-8112000839ad None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:48.890636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b026a-2dba-424c-820d-8112000839ad None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:48.890930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b026a-2dba-424c-820d-8112000839ad None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963/users/7aee709fda854364a32f39ff15c6aafa/roles/aab847245001401cb4d3d43238298f07` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:48.895097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b026a-2dba-424c-820d-8112000839ad None admin] RBAC: Authorizing `identity:create_grant(project_id=e1b269e3c66347e1aa9595854069b963, user_id=7aee709fda854364a32f39ff15c6aafa, role_id=aab847245001401cb4d3d43238298f07)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:48.897339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b026a-2dba-424c-820d-8112000839ad None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:48.913137 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:48.927620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 75/138] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:39:48 2022] PUT /identity/v3/projects/e1b269e3c66347e1aa9595854069b963/users/7aee709fda854364a32f39ff15c6aafa/roles/aab847245001401cb4d3d43238298f07 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:39:48.979262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 64/139] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:49.018770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b026a-2dba-424c-820d-8112000839ad None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:49.167512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b026a-2dba-424c-820d-8112000839ad None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:49.228318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613575f-4dcc-4855-92a8-992c6b8e17e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:49.229364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613575f-4dcc-4855-92a8-992c6b8e17e8 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:49.229708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613575f-4dcc-4855-92a8-992c6b8e17e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:49.230034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613575f-4dcc-4855-92a8-992c6b8e17e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:49.248993 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613575f-4dcc-4855-92a8-992c6b8e17e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-5613575f-4dcc-4855-92a8-992c6b8e17e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:49.258412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:49.259322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:49.260086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 76/140] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:39:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:49.266265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8e001-6982-4a15-b181-ea6449a17c86 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:49.271364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9bc3b-ed18-43f2-820d-85e7c644e028 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:49.271823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9bc3b-ed18-43f2-820d-85e7c644e028 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:49.272360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9bc3b-ed18-43f2-820d-85e7c644e028 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:49.274338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 77/141] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:49.280880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa24d74-a01a-4717-b557-dac3e56ce11b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:49.281213 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa24d74-a01a-4717-b557-dac3e56ce11b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:49.281537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa24d74-a01a-4717-b557-dac3e56ce11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:49.321072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-eaa24d74-a01a-4717-b557-dac3e56ce11b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:49.322327 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-eaa24d74-a01a-4717-b557-dac3e56ce11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:49.395304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:49.457586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 78/142] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:49.475669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8e001-6982-4a15-b181-ea6449a17c86 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:49.513276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa24d74-a01a-4717-b557-dac3e56ce11b None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:49.550599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb77d8-49f2-457f-a59c-e2a6f5178207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:49.551707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb77d8-49f2-457f-a59c-e2a6f5178207 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:49.552226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb77d8-49f2-457f-a59c-e2a6f5178207 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:49.552852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb77d8-49f2-457f-a59c-e2a6f5178207 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:49.554802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb77d8-49f2-457f-a59c-e2a6f5178207 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:49.555383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb77d8-49f2-457f-a59c-e2a6f5178207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:49.558434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb77d8-49f2-457f-a59c-e2a6f5178207 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:49.574700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 65/143] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:49.581953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb77d8-49f2-457f-a59c-e2a6f5178207 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:49.618482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:49.739378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa24d74-a01a-4717-b557-dac3e56ce11b None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:49.780441 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753a8bf-763f-45ae-976f-07f977274941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:49.781306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753a8bf-763f-45ae-976f-07f977274941 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:49.781763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753a8bf-763f-45ae-976f-07f977274941 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:49.782031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753a8bf-763f-45ae-976f-07f977274941 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:49.794646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753a8bf-763f-45ae-976f-07f977274941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:49.800552 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb77d8-49f2-457f-a59c-e2a6f5178207 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-c753a8bf-763f-45ae-976f-07f977274941 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:49.803138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:49.803997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:49.804857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 79/144] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:39:49.813047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753a8bf-763f-45ae-976f-07f977274941 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:49.874329 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:49.889187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dc8c4-c7be-44c8-89c1-403be134b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:49.890105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dc8c4-c7be-44c8-89c1-403be134b606 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:49.890401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dc8c4-c7be-44c8-89c1-403be134b606 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:49.890689 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dc8c4-c7be-44c8-89c1-403be134b606 None admin] PATH_INFO: `/v3/users/adee0d7d3c614774bcfa96ea7aed77e6` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:49.892041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dc8c4-c7be-44c8-89c1-403be134b606 None admin] RBAC: Authorizing `identity:get_user(user_id=adee0d7d3c614774bcfa96ea7aed77e6)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:49.893702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dc8c4-c7be-44c8-89c1-403be134b606 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:49.895475 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 66/145] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:49 2022] GET /identity/v3/users/adee0d7d3c614774bcfa96ea7aed77e6 => generated 247 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:49.901445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dc8c4-c7be-44c8-89c1-403be134b606 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:49.950657 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753a8bf-763f-45ae-976f-07f977274941 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:49.994934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e773c6-1bfc-42c1-a78e-768818f4fa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:49.995627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e773c6-1bfc-42c1-a78e-768818f4fa70 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:49.995907 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e773c6-1bfc-42c1-a78e-768818f4fa70 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:49.996185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e773c6-1bfc-42c1-a78e-768818f4fa70 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:49.996975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e773c6-1bfc-42c1-a78e-768818f4fa70 None admin] RBAC: Adding query filter params (name=member) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:49.997289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e773c6-1bfc-42c1-a78e-768818f4fa70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:49.998711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e773c6-1bfc-42c1-a78e-768818f4fa70 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:50.018667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 80/146] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:39:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:50.020922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2343c0-5a18-48cd-a6ee-26a2872a29d7 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:50.021678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2343c0-5a18-48cd-a6ee-26a2872a29d7 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:50.021678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2343c0-5a18-48cd-a6ee-26a2872a29d7 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:50.026002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 81/147] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:50.026002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2343c0-5a18-48cd-a6ee-26a2872a29d7 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:50.065400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:50.092440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dc8c4-c7be-44c8-89c1-403be134b606 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:50.174330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bce8ea-e34d-4081-bd7a-a3aa2ab7aebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:50.175444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bce8ea-e34d-4081-bd7a-a3aa2ab7aebf None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:50.175929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bce8ea-e34d-4081-bd7a-a3aa2ab7aebf None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:50.176221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bce8ea-e34d-4081-bd7a-a3aa2ab7aebf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:50.271287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:50.284155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2343c0-5a18-48cd-a6ee-26a2872a29d7 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:50.348698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e6a7-c4be-4352-b7ed-d1062a700d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:50.349515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e6a7-c4be-4352-b7ed-d1062a700d65 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:50.350349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e6a7-c4be-4352-b7ed-d1062a700d65 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:50.350628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e6a7-c4be-4352-b7ed-d1062a700d65 None admin] PATH_INFO: `/v3/users/7aee709fda854364a32f39ff15c6aafa` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:50.355274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3e6a7-c4be-4352-b7ed-d1062a700d65 None admin] RBAC: Authorizing `identity:get_user(user_id=7aee709fda854364a32f39ff15c6aafa)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:50.357427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bce8ea-e34d-4081-bd7a-a3aa2ab7aebf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:50.358939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bce8ea-e34d-4081-bd7a-a3aa2ab7aebf None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:50.361377 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3e6a7-c4be-4352-b7ed-d1062a700d65 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:50.363289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 82/148] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:50 2022] GET /identity/v3/users/7aee709fda854364a32f39ff15c6aafa => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:50.364587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ab95-ccb4-4c6b-aa90-613b8ac40ed1 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:50.364873 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ab95-ccb4-4c6b-aa90-613b8ac40ed1 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:50.365195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ab95-ccb4-4c6b-aa90-613b8ac40ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:50.405925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-2388ab95-ccb4-4c6b-aa90-613b8ac40ed1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:50.407177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2388ab95-ccb4-4c6b-aa90-613b8ac40ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:50.489803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 83/149] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:50.490722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388ab95-ccb4-4c6b-aa90-613b8ac40ed1 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:50.639456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388ab95-ccb4-4c6b-aa90-613b8ac40ed1 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:50.681415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a60a7-7ffc-431a-80c7-33526cbbde63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:50.682100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a60a7-7ffc-431a-80c7-33526cbbde63 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:50.682391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a60a7-7ffc-431a-80c7-33526cbbde63 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:50.682668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a60a7-7ffc-431a-80c7-33526cbbde63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:50.688736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:50.698187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:50.755488 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 67/150] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 854 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:39:50.757128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bce8ea-e34d-4081-bd7a-a3aa2ab7aebf None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:50.777691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a60a7-7ffc-431a-80c7-33526cbbde63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:50.779982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a60a7-7ffc-431a-80c7-33526cbbde63 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:50.898825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 84/151] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:39:50.899361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:50.900731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a60a7-7ffc-431a-80c7-33526cbbde63 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:50.978283 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bce8ea-e34d-4081-bd7a-a3aa2ab7aebf None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:51.033896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a60a7-7ffc-431a-80c7-33526cbbde63 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:51.058956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296e238-15cb-45dd-8111-46f53998b67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:51.059770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296e238-15cb-45dd-8111-46f53998b67e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.060057 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296e238-15cb-45dd-8111-46f53998b67e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.060415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296e238-15cb-45dd-8111-46f53998b67e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.065424 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296e238-15cb-45dd-8111-46f53998b67e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:51.066714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296e238-15cb-45dd-8111-46f53998b67e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:51.069059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 68/152] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:51.069951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296e238-15cb-45dd-8111-46f53998b67e None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:51.074573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5099b-638a-400b-a09f-299891962045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:51.075701 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5099b-638a-400b-a09f-299891962045 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.076216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5099b-638a-400b-a09f-299891962045 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.076710 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5099b-638a-400b-a09f-299891962045 None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.078371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5099b-638a-400b-a09f-299891962045 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:51.080048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5099b-638a-400b-a09f-299891962045 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:51.083122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 85/153] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:50 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:51.084784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5099b-638a-400b-a09f-299891962045 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:51.161872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:51.181718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:51.236762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5099b-638a-400b-a09f-299891962045 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:51.259851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296e238-15cb-45dd-8111-46f53998b67e None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:51.274828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1232232-e4cd-43d7-bb1e-48c483cb3ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:51.275872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1232232-e4cd-43d7-bb1e-48c483cb3ebe None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.276396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1232232-e4cd-43d7-bb1e-48c483cb3ebe None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.276903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1232232-e4cd-43d7-bb1e-48c483cb3ebe None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.277822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1232232-e4cd-43d7-bb1e-48c483cb3ebe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:51.279458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1232232-e4cd-43d7-bb1e-48c483cb3ebe None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:51.297831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728264f5-89de-463e-b282-7660efb53b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:51.298733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728264f5-89de-463e-b282-7660efb53b8a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.298990 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728264f5-89de-463e-b282-7660efb53b8a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.299264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728264f5-89de-463e-b282-7660efb53b8a None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.300600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728264f5-89de-463e-b282-7660efb53b8a None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:51.302112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728264f5-89de-463e-b282-7660efb53b8a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:51.304485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 86/154] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:39:51 2022] POST /identity/v3/users => generated 283 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:39:51.304728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 69/155] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:51 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:51.305938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1232232-e4cd-43d7-bb1e-48c483cb3ebe None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:51.313757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf535f-76cb-4969-ab66-569f6610ae54 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.313757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf535f-76cb-4969-ab66-569f6610ae54 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.313757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf535f-76cb-4969-ab66-569f6610ae54 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.314713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 70/156] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:51.315510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf535f-76cb-4969-ab66-569f6610ae54 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:51.432576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1232232-e4cd-43d7-bb1e-48c483cb3ebe None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:51.446765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf535f-76cb-4969-ab66-569f6610ae54 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:51.454971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:51.481461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:51.486919 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb74e4-e3a1-4d5a-91f3-317ee65a2da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:51.487765 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb74e4-e3a1-4d5a-91f3-317ee65a2da4 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.488154 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb74e4-e3a1-4d5a-91f3-317ee65a2da4 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.488572 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb74e4-e3a1-4d5a-91f3-317ee65a2da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.489695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb74e4-e3a1-4d5a-91f3-317ee65a2da4 None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=e1b269e3c66347e1aa9595854069b963, user.id=adee0d7d3c614774bcfa96ea7aed77e6) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:51.490115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb74e4-e3a1-4d5a-91f3-317ee65a2da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:51.492449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb74e4-e3a1-4d5a-91f3-317ee65a2da4 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:51.493395 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5c94a-f0d4-4e8d-8fc2-72125dcf71bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:51.494217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5c94a-f0d4-4e8d-8fc2-72125dcf71bf None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.494751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5c94a-f0d4-4e8d-8fc2-72125dcf71bf None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.495284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5c94a-f0d4-4e8d-8fc2-72125dcf71bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.496492 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5c94a-f0d4-4e8d-8fc2-72125dcf71bf None admin] RBAC: Adding query filter params (role.id=0f8e73b285204863a74f8af4ff7bbc86, scope.project.id=e1b269e3c66347e1aa9595854069b963, user.id=7aee709fda854364a32f39ff15c6aafa) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:51.497004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5c94a-f0d4-4e8d-8fc2-72125dcf71bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:51.498621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5c94a-f0d4-4e8d-8fc2-72125dcf71bf None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:51.499871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 87/157] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:51 2022] GET /identity/v3/role_assignments?user.id=adee0d7d3c614774bcfa96ea7aed77e6&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:51.501381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961b370-edd1-4e54-a60e-7ed61718b7c2 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.501752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961b370-edd1-4e54-a60e-7ed61718b7c2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.502109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961b370-edd1-4e54-a60e-7ed61718b7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.507550 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 71/158] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:51 2022] GET /identity/v3/role_assignments?user.id=7aee709fda854364a32f39ff15c6aafa&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:51.522672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-1961b370-edd1-4e54-a60e-7ed61718b7c2 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:51.524057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1961b370-edd1-4e54-a60e-7ed61718b7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:51.607929 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 88/159] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:51.633351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5c94a-f0d4-4e8d-8fc2-72125dcf71bf None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:51.809997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5c94a-f0d4-4e8d-8fc2-72125dcf71bf None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:51.833079 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aba0ef-6681-4cbd-b617-f9825f554021 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.833399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aba0ef-6681-4cbd-b617-f9825f554021 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.833641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aba0ef-6681-4cbd-b617-f9825f554021 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.835641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 89/160] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:39:51.842905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dfa1c-33f9-4fb1-8ff8-062d7ea6e312 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.843197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dfa1c-33f9-4fb1-8ff8-062d7ea6e312 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.843479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dfa1c-33f9-4fb1-8ff8-062d7ea6e312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.861610 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33ba36-1291-4318-b162-f6cfda708647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:51.862431 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33ba36-1291-4318-b162-f6cfda708647 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.862809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33ba36-1291-4318-b162-f6cfda708647 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.863092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33ba36-1291-4318-b162-f6cfda708647 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.869694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-a81dfa1c-33f9-4fb1-8ff8-062d7ea6e312 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:51.870933 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a81dfa1c-33f9-4fb1-8ff8-062d7ea6e312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:51.877095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33ba36-1291-4318-b162-f6cfda708647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-6f33ba36-1291-4318-b162-f6cfda708647 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:51.886176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:51.887079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:39:51.887878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 72/161] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:51.894228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33ba36-1291-4318-b162-f6cfda708647 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:51.971688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 90/162] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:39:51.974085 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:51.988277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f59ff1-cdbb-41df-a462-8e4c462b39a4 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.988779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f59ff1-cdbb-41df-a462-8e4c462b39a4 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.989061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f59ff1-cdbb-41df-a462-8e4c462b39a4 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:51.990980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 91/163] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:39:51.997832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1924e-518f-4c7c-b368-63f862e2e98a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:51.998124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1924e-518f-4c7c-b368-63f862e2e98a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:51.998410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1924e-518f-4c7c-b368-63f862e2e98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:52.027114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-90e1924e-518f-4c7c-b368-63f862e2e98a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:52.028412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-90e1924e-518f-4c7c-b368-63f862e2e98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:52.053166 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33ba36-1291-4318-b162-f6cfda708647 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:52.092796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:52.097326 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e69686-f787-446f-a5b2-feae1cd51548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:52.098673 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e69686-f787-446f-a5b2-feae1cd51548 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:52.098973 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e69686-f787-446f-a5b2-feae1cd51548 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:52.099250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e69686-f787-446f-a5b2-feae1cd51548 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:52.100060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e69686-f787-446f-a5b2-feae1cd51548 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:52.100497 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e69686-f787-446f-a5b2-feae1cd51548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:52.102060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e69686-f787-446f-a5b2-feae1cd51548 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:52.116989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 73/164] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:39:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:52.118184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e69686-f787-446f-a5b2-feae1cd51548 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:52.141909 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 92/165] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:52.142881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1924e-518f-4c7c-b368-63f862e2e98a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:52.295477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1924e-518f-4c7c-b368-63f862e2e98a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:52.314071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e69686-f787-446f-a5b2-feae1cd51548 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:52.342358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854775b-3513-49aa-98a5-1a79a7e9aaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:52.343643 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854775b-3513-49aa-98a5-1a79a7e9aaec None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:52.343875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854775b-3513-49aa-98a5-1a79a7e9aaec None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:52.344139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854775b-3513-49aa-98a5-1a79a7e9aaec None admin] PATH_INFO: `/v3/users/b3e729e34c514993aae340dcfaf4e1d8` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:52.345460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854775b-3513-49aa-98a5-1a79a7e9aaec None admin] RBAC: Authorizing `identity:get_user(user_id=b3e729e34c514993aae340dcfaf4e1d8)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:52.347014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854775b-3513-49aa-98a5-1a79a7e9aaec None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:52.349124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 93/166] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:52 2022] GET /identity/v3/users/b3e729e34c514993aae340dcfaf4e1d8 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:52.350448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854775b-3513-49aa-98a5-1a79a7e9aaec None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:52.350775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:52.368896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a78b89-e2d1-49ba-b017-51b1c136e99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:52.369716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a78b89-e2d1-49ba-b017-51b1c136e99b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:52.370278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a78b89-e2d1-49ba-b017-51b1c136e99b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:52.370570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a78b89-e2d1-49ba-b017-51b1c136e99b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:52.386784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a78b89-e2d1-49ba-b017-51b1c136e99b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-d6a78b89-e2d1-49ba-b017-51b1c136e99b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:52.397958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:52.399896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 74/167] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:52.402145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a78b89-e2d1-49ba-b017-51b1c136e99b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:52.406376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:52.476777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854775b-3513-49aa-98a5-1a79a7e9aaec None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:52.529603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e1e00-43a0-466c-94d4-0cb366ee8327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:52.530438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e1e00-43a0-466c-94d4-0cb366ee8327 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:52.530726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e1e00-43a0-466c-94d4-0cb366ee8327 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:52.531007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e1e00-43a0-466c-94d4-0cb366ee8327 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:52.540492 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a78b89-e2d1-49ba-b017-51b1c136e99b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:52.565281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e1e00-43a0-466c-94d4-0cb366ee8327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:52.579128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b6b6f-2a70-4681-b3fa-ccfab8199fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:52.580098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b6b6f-2a70-4681-b3fa-ccfab8199fbe None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:52.580486 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b6b6f-2a70-4681-b3fa-ccfab8199fbe None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:52.580861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b6b6f-2a70-4681-b3fa-ccfab8199fbe None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:52.582187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b6b6f-2a70-4681-b3fa-ccfab8199fbe None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:52.583723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b6b6f-2a70-4681-b3fa-ccfab8199fbe None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:52.585923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 75/168] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:52 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:52.586789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b6b6f-2a70-4681-b3fa-ccfab8199fbe None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-cb9e1e00-43a0-466c-94d4-0cb366ee8327 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:52.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:52.588773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:52.589581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 94/169] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:39:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:52.596339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e1e00-43a0-466c-94d4-0cb366ee8327 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:52.645376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:52.722518 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b6b6f-2a70-4681-b3fa-ccfab8199fbe None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:52.776659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:52.780679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e1e00-43a0-466c-94d4-0cb366ee8327 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:52.789838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55eec0-2c67-425d-9d08-2421b81b53e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:52.790580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55eec0-2c67-425d-9d08-2421b81b53e6 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:52.790863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55eec0-2c67-425d-9d08-2421b81b53e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:52.791195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55eec0-2c67-425d-9d08-2421b81b53e6 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:52.792099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55eec0-2c67-425d-9d08-2421b81b53e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:52.792432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55eec0-2c67-425d-9d08-2421b81b53e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:52.795081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55eec0-2c67-425d-9d08-2421b81b53e6 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:52.813801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 76/170] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:39:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:52.815918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55eec0-2c67-425d-9d08-2421b81b53e6 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:52.852538 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d4f8-45f8-47c7-91e1-f4dbe4bef7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:52.853771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d4f8-45f8-47c7-91e1-f4dbe4bef7cd None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:52.854056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d4f8-45f8-47c7-91e1-f4dbe4bef7cd None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:52.854360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d4f8-45f8-47c7-91e1-f4dbe4bef7cd None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963/users/b3e729e34c514993aae340dcfaf4e1d8/roles/2cfac7d7d3f040f2a24b48e4f01d3eb2` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:52.859845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913d4f8-45f8-47c7-91e1-f4dbe4bef7cd None admin] RBAC: Authorizing `identity:create_grant(project_id=e1b269e3c66347e1aa9595854069b963, user_id=b3e729e34c514993aae340dcfaf4e1d8, role_id=2cfac7d7d3f040f2a24b48e4f01d3eb2)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:52.864191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913d4f8-45f8-47c7-91e1-f4dbe4bef7cd None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:52.885141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 95/171] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:39:52 2022] PUT /identity/v3/projects/e1b269e3c66347e1aa9595854069b963/users/b3e729e34c514993aae340dcfaf4e1d8/roles/2cfac7d7d3f040f2a24b48e4f01d3eb2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:39:52.886953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d4f8-45f8-47c7-91e1-f4dbe4bef7cd None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:53.080108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55eec0-2c67-425d-9d08-2421b81b53e6 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:53.084596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:53.161904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d4f8-45f8-47c7-91e1-f4dbe4bef7cd None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:53.178177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63964484-5618-4f22-b462-728cd37e655e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:53.178869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63964484-5618-4f22-b462-728cd37e655e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:53.179150 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63964484-5618-4f22-b462-728cd37e655e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:53.179424 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63964484-5618-4f22-b462-728cd37e655e None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:53.180188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63964484-5618-4f22-b462-728cd37e655e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:53.180621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63964484-5618-4f22-b462-728cd37e655e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:53.182262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63964484-5618-4f22-b462-728cd37e655e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:53.191802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 77/172] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:39:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:53.201091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63964484-5618-4f22-b462-728cd37e655e None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:53.213092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61a764-77b9-473d-8e8c-cddf5a16d5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:53.213999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61a764-77b9-473d-8e8c-cddf5a16d5f1 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:53.214286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61a764-77b9-473d-8e8c-cddf5a16d5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:53.214623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61a764-77b9-473d-8e8c-cddf5a16d5f1 None admin] PATH_INFO: `/v3/users/7aee709fda854364a32f39ff15c6aafa` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:53.224117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61a764-77b9-473d-8e8c-cddf5a16d5f1 None admin] RBAC: Authorizing `identity:get_user(user_id=7aee709fda854364a32f39ff15c6aafa)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:53.224117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61a764-77b9-473d-8e8c-cddf5a16d5f1 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:53.224117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 96/173] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:52 2022] GET /identity/v3/users/7aee709fda854364a32f39ff15c6aafa => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:53.231204 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61a764-77b9-473d-8e8c-cddf5a16d5f1 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:53.301412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:53.447449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63964484-5618-4f22-b462-728cd37e655e None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:53.520251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61a764-77b9-473d-8e8c-cddf5a16d5f1 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:53.539460 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3f3be-70bf-4008-a3c5-ee5fe4ef3724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:53.540435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3f3be-70bf-4008-a3c5-ee5fe4ef3724 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:53.540744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3f3be-70bf-4008-a3c5-ee5fe4ef3724 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:53.541356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3f3be-70bf-4008-a3c5-ee5fe4ef3724 None admin] PATH_INFO: `/v3/users/7aee709fda854364a32f39ff15c6aafa` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:53.544791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3f3be-70bf-4008-a3c5-ee5fe4ef3724 None admin] RBAC: Authorizing `identity:get_user(user_id=7aee709fda854364a32f39ff15c6aafa)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:53.546290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3f3be-70bf-4008-a3c5-ee5fe4ef3724 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:53.548302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 78/174] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:53 2022] GET /identity/v3/users/7aee709fda854364a32f39ff15c6aafa => generated 275 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:53.557086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3f3be-70bf-4008-a3c5-ee5fe4ef3724 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:53.612836 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a1e52-022e-4b81-9ee1-2f2904ad5aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:53.614260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a1e52-022e-4b81-9ee1-2f2904ad5aae None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:53.614830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a1e52-022e-4b81-9ee1-2f2904ad5aae None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:53.615838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a1e52-022e-4b81-9ee1-2f2904ad5aae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:53.630660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:53.758048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3f3be-70bf-4008-a3c5-ee5fe4ef3724 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:53.798672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a1e52-022e-4b81-9ee1-2f2904ad5aae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:53.800699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a1e52-022e-4b81-9ee1-2f2904ad5aae None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:53.814486 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6dbd9-99ba-40cb-b62b-425959f9a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:53.815225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6dbd9-99ba-40cb-b62b-425959f9a017 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:53.815515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6dbd9-99ba-40cb-b62b-425959f9a017 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:53.815795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6dbd9-99ba-40cb-b62b-425959f9a017 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:53.861396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:53.957174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6dbd9-99ba-40cb-b62b-425959f9a017 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:53.958666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6dbd9-99ba-40cb-b62b-425959f9a017 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:54.006277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:54.041391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 97/175] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 810 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:39:54.047455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a1e52-022e-4b81-9ee1-2f2904ad5aae None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:54.138690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 79/176] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:39:54.144397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6dbd9-99ba-40cb-b62b-425959f9a017 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:54.287403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a1e52-022e-4b81-9ee1-2f2904ad5aae None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:54.344693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:54.364020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9071c4-bb08-466a-a126-94aabed512f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:54.365686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9071c4-bb08-466a-a126-94aabed512f6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:54.366015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9071c4-bb08-466a-a126-94aabed512f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:54.366296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9071c4-bb08-466a-a126-94aabed512f6 None admin] PATH_INFO: `/v3/projects/7fc29e50692c4f1a866578fee9aa0044` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:54.369144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9071c4-bb08-466a-a126-94aabed512f6 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc29e50692c4f1a866578fee9aa0044)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:54.370628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9071c4-bb08-466a-a126-94aabed512f6 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:54.372685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 98/177] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:54 2022] GET /identity/v3/projects/7fc29e50692c4f1a866578fee9aa0044 => generated 312 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:54.378584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9071c4-bb08-466a-a126-94aabed512f6 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:54.392765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6dbd9-99ba-40cb-b62b-425959f9a017 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:54.470465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:54.478547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cc434-394a-486c-8b2f-db562c4778be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:54.479554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cc434-394a-486c-8b2f-db562c4778be None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:54.479849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cc434-394a-486c-8b2f-db562c4778be None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:54.480129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cc434-394a-486c-8b2f-db562c4778be None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:54.481510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cc434-394a-486c-8b2f-db562c4778be None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:54.485268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cc434-394a-486c-8b2f-db562c4778be None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:54.487336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 80/178] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:54 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:54.500764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cc434-394a-486c-8b2f-db562c4778be None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:54.586906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9071c4-bb08-466a-a126-94aabed512f6 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:54.654351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf24f99-b8a1-4cdf-a160-abbce89d9460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:54.655351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24f99-b8a1-4cdf-a160-abbce89d9460 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:54.655813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24f99-b8a1-4cdf-a160-abbce89d9460 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:54.656261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf24f99-b8a1-4cdf-a160-abbce89d9460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:54.657737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf24f99-b8a1-4cdf-a160-abbce89d9460 None admin] RBAC: Adding query filter params (role.id=0f8e73b285204863a74f8af4ff7bbc86, scope.project.id=7fc29e50692c4f1a866578fee9aa0044, user.id=7aee709fda854364a32f39ff15c6aafa) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:54.667056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf24f99-b8a1-4cdf-a160-abbce89d9460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:54.669030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf24f99-b8a1-4cdf-a160-abbce89d9460 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:54.681858 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 99/179] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:54 2022] GET /identity/v3/role_assignments?user.id=7aee709fda854364a32f39ff15c6aafa&scope.project.id=7fc29e50692c4f1a866578fee9aa0044&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:54.718131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cc434-394a-486c-8b2f-db562c4778be None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:54.778666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea82008-041a-45ad-8cf0-f52377c5f996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:54.779728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82008-041a-45ad-8cf0-f52377c5f996 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:54.780207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82008-041a-45ad-8cf0-f52377c5f996 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:54.780708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82008-041a-45ad-8cf0-f52377c5f996 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:54.781797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea82008-041a-45ad-8cf0-f52377c5f996 None admin] RBAC: Adding query filter params (role.id=aab847245001401cb4d3d43238298f07, scope.project.id=e1b269e3c66347e1aa9595854069b963, user.id=7aee709fda854364a32f39ff15c6aafa) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:54.782333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea82008-041a-45ad-8cf0-f52377c5f996 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:54.784135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea82008-041a-45ad-8cf0-f52377c5f996 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:54.797839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 81/180] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:54 2022] GET /identity/v3/role_assignments?user.id=7aee709fda854364a32f39ff15c6aafa&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=aab847245001401cb4d3d43238298f07 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:54.843037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0849c6e-079e-4305-890f-7b0d54a0704c None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:54.843376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0849c6e-079e-4305-890f-7b0d54a0704c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:54.843594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0849c6e-079e-4305-890f-7b0d54a0704c None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:54.845237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 100/181] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:54.862180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e2cda-ccad-4808-9976-b9fe52322e68 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:54.862812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e2cda-ccad-4808-9976-b9fe52322e68 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:54.863311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e2cda-ccad-4808-9976-b9fe52322e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:54.893532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-315e2cda-ccad-4808-9976-b9fe52322e68 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:54.895124 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-315e2cda-ccad-4808-9976-b9fe52322e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:54.982158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:54.994177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 82/182] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:55.040167 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0849c6e-079e-4305-890f-7b0d54a0704c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:55.087354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0375-ff61-43c5-9a6b-3ac0fd9ff7a0 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.088255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0375-ff61-43c5-9a6b-3ac0fd9ff7a0 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.088752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0375-ff61-43c5-9a6b-3ac0fd9ff7a0 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.091351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 83/183] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:55.098056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532d0ac-3d24-41c9-9684-36c44c119e2a None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.098850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532d0ac-3d24-41c9-9684-36c44c119e2a None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.100809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532d0ac-3d24-41c9-9684-36c44c119e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.122456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-f532d0ac-3d24-41c9-9684-36c44c119e2a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:55.123677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f532d0ac-3d24-41c9-9684-36c44c119e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:55.136398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:55.196310 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0849c6e-079e-4305-890f-7b0d54a0704c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:55.217564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 84/184] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:39:55.219241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e91a61-2d26-44c1-aa2a-afd98178336d None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.219797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e91a61-2d26-44c1-aa2a-afd98178336d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.220079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e91a61-2d26-44c1-aa2a-afd98178336d None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.227901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 85/185] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:55.229903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddc223-1400-480d-b6e3-5bdc1d415304 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.230344 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddc223-1400-480d-b6e3-5bdc1d415304 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.230767 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddc223-1400-480d-b6e3-5bdc1d415304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.252103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-96ddc223-1400-480d-b6e3-5bdc1d415304 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:55.253661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-96ddc223-1400-480d-b6e3-5bdc1d415304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:55.255404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47a8f5-3f67-415b-9034-d720d9811279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:55.259033 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a8f5-3f67-415b-9034-d720d9811279 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.259501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a8f5-3f67-415b-9034-d720d9811279 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.260041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a8f5-3f67-415b-9034-d720d9811279 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.273923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47a8f5-3f67-415b-9034-d720d9811279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-da47a8f5-3f67-415b-9034-d720d9811279 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:55.281769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:55.282686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:55.283469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 101/186] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:39:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:55.284159 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47a8f5-3f67-415b-9034-d720d9811279 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:55.342827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 86/187] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:39:55.343568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddc223-1400-480d-b6e3-5bdc1d415304 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:55.410011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:55.417208 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47a8f5-3f67-415b-9034-d720d9811279 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:55.438247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:55.454065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527337a8-0a4c-4d0f-a0c8-e67c1cc00e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:55.454870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527337a8-0a4c-4d0f-a0c8-e67c1cc00e4d None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.455154 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527337a8-0a4c-4d0f-a0c8-e67c1cc00e4d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.455430 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527337a8-0a4c-4d0f-a0c8-e67c1cc00e4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.456685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527337a8-0a4c-4d0f-a0c8-e67c1cc00e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:55.458149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527337a8-0a4c-4d0f-a0c8-e67c1cc00e4d None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:55.464785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 102/188] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:55.464942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddc223-1400-480d-b6e3-5bdc1d415304 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:55.465021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527337a8-0a4c-4d0f-a0c8-e67c1cc00e4d None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:55.507190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be20fa-902e-4942-8e18-64c7fac7ddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:55.508419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be20fa-902e-4942-8e18-64c7fac7ddef None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.508715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be20fa-902e-4942-8e18-64c7fac7ddef None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.509005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be20fa-902e-4942-8e18-64c7fac7ddef None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.509774 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be20fa-902e-4942-8e18-64c7fac7ddef None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:55.510084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be20fa-902e-4942-8e18-64c7fac7ddef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:55.511481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be20fa-902e-4942-8e18-64c7fac7ddef None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:55.521470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 87/189] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:55.522510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be20fa-902e-4942-8e18-64c7fac7ddef None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:55.581444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527337a8-0a4c-4d0f-a0c8-e67c1cc00e4d None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:55.621332 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51a5be-b8d3-4c82-ba76-05ffcc1676f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:55.622145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51a5be-b8d3-4c82-ba76-05ffcc1676f2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.622430 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51a5be-b8d3-4c82-ba76-05ffcc1676f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.622855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51a5be-b8d3-4c82-ba76-05ffcc1676f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.635339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51a5be-b8d3-4c82-ba76-05ffcc1676f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:55.638964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be20fa-902e-4942-8e18-64c7fac7ddef None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-7a51a5be-b8d3-4c82-ba76-05ffcc1676f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:55.646168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:55.647726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 103/190] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:39:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:55.654321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51a5be-b8d3-4c82-ba76-05ffcc1676f2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:55.672513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:55.697980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c9287-8822-4e9a-bf68-9485e9cb51b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:55.698659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c9287-8822-4e9a-bf68-9485e9cb51b3 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.698964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c9287-8822-4e9a-bf68-9485e9cb51b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.699216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c9287-8822-4e9a-bf68-9485e9cb51b3 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.699935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c9287-8822-4e9a-bf68-9485e9cb51b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:55.701435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c9287-8822-4e9a-bf68-9485e9cb51b3 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:55.731420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:55.734787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 88/191] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:39:55 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:39:55.735735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c9287-8822-4e9a-bf68-9485e9cb51b3 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:55.806813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51a5be-b8d3-4c82-ba76-05ffcc1676f2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:55.856269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc55a46-e35f-4757-a5f2-7a171299823c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:55.857209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc55a46-e35f-4757-a5f2-7a171299823c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.857492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc55a46-e35f-4757-a5f2-7a171299823c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.857767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc55a46-e35f-4757-a5f2-7a171299823c None admin] PATH_INFO: `/v3/users/6caffb9893c44b68a67f8c44e43db934` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.881212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc55a46-e35f-4757-a5f2-7a171299823c None admin] RBAC: Authorizing `identity:get_user(user_id=6caffb9893c44b68a67f8c44e43db934)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:55.882769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc55a46-e35f-4757-a5f2-7a171299823c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:55.889129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 104/192] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:55 2022] GET /identity/v3/users/6caffb9893c44b68a67f8c44e43db934 => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:55.889129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36878057-a333-40bb-b095-6348bf17219f None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.889129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36878057-a333-40bb-b095-6348bf17219f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.889129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36878057-a333-40bb-b095-6348bf17219f None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.889129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 105/193] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:55.891915 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36878057-a333-40bb-b095-6348bf17219f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:55.898224 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c9287-8822-4e9a-bf68-9485e9cb51b3 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:55.943822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735c41f-8b27-4dd6-a070-a400a7446792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:55.945019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735c41f-8b27-4dd6-a070-a400a7446792 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.945347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735c41f-8b27-4dd6-a070-a400a7446792 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.945782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735c41f-8b27-4dd6-a070-a400a7446792 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.947608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735c41f-8b27-4dd6-a070-a400a7446792 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:55.948111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735c41f-8b27-4dd6-a070-a400a7446792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:55.951087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735c41f-8b27-4dd6-a070-a400a7446792 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:55.968443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 89/194] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:39:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:55.970751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27aeb2-f301-44cd-a694-201d27a90bb9 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:55.971061 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27aeb2-f301-44cd-a694-201d27a90bb9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:55.971621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27aeb2-f301-44cd-a694-201d27a90bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:55.996863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ec27aeb2-f301-44cd-a694-201d27a90bb9 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:55.998111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ec27aeb2-f301-44cd-a694-201d27a90bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:56.033009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36878057-a333-40bb-b095-6348bf17219f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:56.034877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:56.044419 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:56.083608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fd71b-feb0-42f9-94e2-a478179c5a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:56.084464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fd71b-feb0-42f9-94e2-a478179c5a29 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:56.084617 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fd71b-feb0-42f9-94e2-a478179c5a29 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:56.084935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fd71b-feb0-42f9-94e2-a478179c5a29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:56.106094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 90/195] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:39:56.107034 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27aeb2-f301-44cd-a694-201d27a90bb9 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:56.215352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fd71b-feb0-42f9-94e2-a478179c5a29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:56.217043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fd71b-feb0-42f9-94e2-a478179c5a29 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:56.236099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27aeb2-f301-44cd-a694-201d27a90bb9 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:56.273723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ca35e-0b3f-426c-820c-92798b81d651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:56.274983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ca35e-0b3f-426c-820c-92798b81d651 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:56.275471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ca35e-0b3f-426c-820c-92798b81d651 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:56.275957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ca35e-0b3f-426c-820c-92798b81d651 None admin] PATH_INFO: `/v3/users/6caffb9893c44b68a67f8c44e43db934` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:56.277582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ca35e-0b3f-426c-820c-92798b81d651 None admin] RBAC: Authorizing `identity:get_user(user_id=6caffb9893c44b68a67f8c44e43db934)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:56.279282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ca35e-0b3f-426c-820c-92798b81d651 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:56.281569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 91/196] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:56 2022] GET /identity/v3/users/6caffb9893c44b68a67f8c44e43db934 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:56.282681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ca35e-0b3f-426c-820c-92798b81d651 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:56.321177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:56.340011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 106/197] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:39:56.341377 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fd71b-feb0-42f9-94e2-a478179c5a29 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:56.352732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:56.411842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ca35e-0b3f-426c-820c-92798b81d651 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:56.449692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29457b9-771f-49e4-b3ef-516b326e60c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:56.450900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29457b9-771f-49e4-b3ef-516b326e60c9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:56.451383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29457b9-771f-49e4-b3ef-516b326e60c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:56.451861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29457b9-771f-49e4-b3ef-516b326e60c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:56.460834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fd71b-feb0-42f9-94e2-a478179c5a29 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:56.464879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29457b9-771f-49e4-b3ef-516b326e60c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-a29457b9-771f-49e4-b3ef-516b326e60c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:56.473254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:56.474156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:39:56.475087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:39:56.477540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 92/198] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:56.478563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29457b9-771f-49e4-b3ef-516b326e60c9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:56.517510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f6eb7-cbe9-43de-bdc7-0b976790d1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:56.518207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f6eb7-cbe9-43de-bdc7-0b976790d1bf None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:56.518489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f6eb7-cbe9-43de-bdc7-0b976790d1bf None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:56.518767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f6eb7-cbe9-43de-bdc7-0b976790d1bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:56.601363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f6eb7-cbe9-43de-bdc7-0b976790d1bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:56.603813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:56.606185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f6eb7-cbe9-43de-bdc7-0b976790d1bf None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:56.614181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29457b9-771f-49e4-b3ef-516b326e60c9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:56.625605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:56.670138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b0b6f-2607-4ed2-b69c-0b5ff82c7666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:56.670606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b0b6f-2607-4ed2-b69c-0b5ff82c7666 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:56.670900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b0b6f-2607-4ed2-b69c-0b5ff82c7666 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:56.671181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b0b6f-2607-4ed2-b69c-0b5ff82c7666 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:56.672473 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b0b6f-2607-4ed2-b69c-0b5ff82c7666 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:56.673944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b0b6f-2607-4ed2-b69c-0b5ff82c7666 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:56.676355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 93/199] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:56 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:56.676829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b0b6f-2607-4ed2-b69c-0b5ff82c7666 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:56.791000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b0b6f-2607-4ed2-b69c-0b5ff82c7666 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:56.794740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 107/200] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:39:56.796607 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f6eb7-cbe9-43de-bdc7-0b976790d1bf None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:56.839584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca73ff5-5e10-4f20-a42c-55c8e42e9e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:56.840368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca73ff5-5e10-4f20-a42c-55c8e42e9e02 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:56.840636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca73ff5-5e10-4f20-a42c-55c8e42e9e02 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:56.840954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca73ff5-5e10-4f20-a42c-55c8e42e9e02 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:56.841725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca73ff5-5e10-4f20-a42c-55c8e42e9e02 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:56.842041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca73ff5-5e10-4f20-a42c-55c8e42e9e02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:56.843487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca73ff5-5e10-4f20-a42c-55c8e42e9e02 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:56.856613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 94/201] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:39:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:56.857622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca73ff5-5e10-4f20-a42c-55c8e42e9e02 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:56.887056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:56.955460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f6eb7-cbe9-43de-bdc7-0b976790d1bf None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:56.965117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:56.992935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca73ff5-5e10-4f20-a42c-55c8e42e9e02 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:57.000784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1881bf-b32a-4803-893d-416ea2ae97a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:57.001930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1881bf-b32a-4803-893d-416ea2ae97a5 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:57.002433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1881bf-b32a-4803-893d-416ea2ae97a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:57.002852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1881bf-b32a-4803-893d-416ea2ae97a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:57.003851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1881bf-b32a-4803-893d-416ea2ae97a5 None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=4060784cbaea460b93f362ae4cf23508, user.id=6caffb9893c44b68a67f8c44e43db934) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:57.004345 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1881bf-b32a-4803-893d-416ea2ae97a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:57.012213 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1881bf-b32a-4803-893d-416ea2ae97a5 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:57.023865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 108/202] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:56 2022] GET /identity/v3/role_assignments?user.id=6caffb9893c44b68a67f8c44e43db934&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:57.026038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1881bf-b32a-4803-893d-416ea2ae97a5 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:57.039977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2da711-91f9-48c7-8f6b-22e4b24f270f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:57.040919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2da711-91f9-48c7-8f6b-22e4b24f270f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:57.041212 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2da711-91f9-48c7-8f6b-22e4b24f270f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:57.041491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2da711-91f9-48c7-8f6b-22e4b24f270f None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:57.042917 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2da711-91f9-48c7-8f6b-22e4b24f270f None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:57.050240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2da711-91f9-48c7-8f6b-22e4b24f270f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:57.050240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 95/203] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:56 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:57.057937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2da711-91f9-48c7-8f6b-22e4b24f270f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:57.252156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:57.259548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1881bf-b32a-4803-893d-416ea2ae97a5 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:57.288466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2da711-91f9-48c7-8f6b-22e4b24f270f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:57.345754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a80bef-ade2-493b-9330-37fa05ffb7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:57.346912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80bef-ade2-493b-9330-37fa05ffb7a2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:57.347195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80bef-ade2-493b-9330-37fa05ffb7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:57.347473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80bef-ade2-493b-9330-37fa05ffb7a2 None admin] PATH_INFO: `/v3/users/b3e729e34c514993aae340dcfaf4e1d8` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:57.349371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a80bef-ade2-493b-9330-37fa05ffb7a2 None admin] RBAC: Authorizing `identity:get_user(user_id=b3e729e34c514993aae340dcfaf4e1d8)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:57.351161 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a80bef-ade2-493b-9330-37fa05ffb7a2 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:57.353765 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 109/204] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:57 2022] GET /identity/v3/users/b3e729e34c514993aae340dcfaf4e1d8 => generated 289 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:57.358746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b0615-cb04-40d3-b3d2-99445c49977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:57.359687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b0615-cb04-40d3-b3d2-99445c49977b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:57.360064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b0615-cb04-40d3-b3d2-99445c49977b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:57.364416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b0615-cb04-40d3-b3d2-99445c49977b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:57.364416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b0615-cb04-40d3-b3d2-99445c49977b None admin] RBAC: Adding query filter params (role.id=aab847245001401cb4d3d43238298f07, scope.project.id=4060784cbaea460b93f362ae4cf23508, user.id=6caffb9893c44b68a67f8c44e43db934) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:57.364416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b0615-cb04-40d3-b3d2-99445c49977b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:57.364416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b0615-cb04-40d3-b3d2-99445c49977b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:57.366685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a80bef-ade2-493b-9330-37fa05ffb7a2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:57.376358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 96/205] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:57 2022] GET /identity/v3/role_assignments?user.id=6caffb9893c44b68a67f8c44e43db934&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=aab847245001401cb4d3d43238298f07 => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:57.493245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:57.603733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a80bef-ade2-493b-9330-37fa05ffb7a2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:57.665912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5411f-961f-4b78-9a7a-f21d8e7c3c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:57.666724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5411f-961f-4b78-9a7a-f21d8e7c3c2c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:57.667039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5411f-961f-4b78-9a7a-f21d8e7c3c2c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:57.667355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5411f-961f-4b78-9a7a-f21d8e7c3c2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:57.808148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5411f-961f-4b78-9a7a-f21d8e7c3c2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:57.809620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5411f-961f-4b78-9a7a-f21d8e7c3c2c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:57.887528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:57.976560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 110/206] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:39:57.982336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b0615-cb04-40d3-b3d2-99445c49977b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:58.126454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b0615-cb04-40d3-b3d2-99445c49977b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:58.165316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160adae1-cee0-4406-9687-bef2e50ddaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:58.166533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160adae1-cee0-4406-9687-bef2e50ddaf4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:58.166994 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160adae1-cee0-4406-9687-bef2e50ddaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:58.167441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160adae1-cee0-4406-9687-bef2e50ddaf4 None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:58.169062 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160adae1-cee0-4406-9687-bef2e50ddaf4 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:58.170718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160adae1-cee0-4406-9687-bef2e50ddaf4 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:58.173096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 97/207] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:57 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:58.180353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5411f-961f-4b78-9a7a-f21d8e7c3c2c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:58.356615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5411f-961f-4b78-9a7a-f21d8e7c3c2c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:58.409020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c7746-da85-44db-bea8-3048c86169a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:58.410052 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c7746-da85-44db-bea8-3048c86169a1 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:58.410535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c7746-da85-44db-bea8-3048c86169a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:58.411030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c7746-da85-44db-bea8-3048c86169a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:58.413081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c7746-da85-44db-bea8-3048c86169a1 None admin] RBAC: Adding query filter params (role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2, scope.project.id=e1b269e3c66347e1aa9595854069b963, user.id=b3e729e34c514993aae340dcfaf4e1d8) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:58.413402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c7746-da85-44db-bea8-3048c86169a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:58.414373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987096e-4e20-4749-81ea-d1902b138705 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:58.414879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c7746-da85-44db-bea8-3048c86169a1 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:58.414984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987096e-4e20-4749-81ea-d1902b138705 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:58.415442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987096e-4e20-4749-81ea-d1902b138705 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:58.417319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 98/208] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:39:58.422136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 111/209] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:58 2022] GET /identity/v3/role_assignments?user.id=b3e729e34c514993aae340dcfaf4e1d8&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:58.423436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fa84a-111e-4d65-870b-79ed8a8de0fe None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:58.423933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fa84a-111e-4d65-870b-79ed8a8de0fe None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:58.424447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fa84a-111e-4d65-870b-79ed8a8de0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:58.465690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-1c8fa84a-111e-4d65-870b-79ed8a8de0fe None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:58.467313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1c8fa84a-111e-4d65-870b-79ed8a8de0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:58.481377 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1fe40-62e8-43b2-9a8c-d36fc9f36646 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:58.481703 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1fe40-62e8-43b2-9a8c-d36fc9f36646 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:58.481950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1fe40-62e8-43b2-9a8c-d36fc9f36646 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:58.483625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 112/210] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:39:58.492605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97b906-7663-4d2b-b711-8b69894a625d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:58.492946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97b906-7663-4d2b-b711-8b69894a625d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:58.493158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97b906-7663-4d2b-b711-8b69894a625d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:58.497305 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:58.505618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:58.525497 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-da97b906-7663-4d2b-b711-8b69894a625d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:39:58.526786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-da97b906-7663-4d2b-b711-8b69894a625d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:39:58.574038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 99/211] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:39:58.606982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fa84a-111e-4d65-870b-79ed8a8de0fe None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:58.666491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 113/212] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 19 20:39:58.705235 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97b906-7663-4d2b-b711-8b69894a625d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:58.758206 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fa84a-111e-4d65-870b-79ed8a8de0fe None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:58.831463 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b820c-4a9a-445b-9725-a73442e5c1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:58.833070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b820c-4a9a-445b-9725-a73442e5c1f1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:58.834422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b820c-4a9a-445b-9725-a73442e5c1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:58.835197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b820c-4a9a-445b-9725-a73442e5c1f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:58.855249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b820c-4a9a-445b-9725-a73442e5c1f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-f61b820c-4a9a-445b-9725-a73442e5c1f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:39:58.868172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:39:58.869598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:39:58.870772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:39:58.877767 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 100/213] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:39:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:39:58.881486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97b906-7663-4d2b-b711-8b69894a625d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:58.886218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b820c-4a9a-445b-9725-a73442e5c1f1 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:58.909712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:58.927734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:58.936386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d21928-be31-4fad-bc28-a1ab3861ecbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:58.937405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d21928-be31-4fad-bc28-a1ab3861ecbf None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:58.937796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d21928-be31-4fad-bc28-a1ab3861ecbf None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:58.938138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d21928-be31-4fad-bc28-a1ab3861ecbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:58.939757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d21928-be31-4fad-bc28-a1ab3861ecbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:58.942771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d21928-be31-4fad-bc28-a1ab3861ecbf None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:58.944568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 114/214] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:39:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:58.951306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d21928-be31-4fad-bc28-a1ab3861ecbf None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:59.029943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b820c-4a9a-445b-9725-a73442e5c1f1 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:59.068988 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26a748-5758-4633-b72e-9bc56f199281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:59.069642 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26a748-5758-4633-b72e-9bc56f199281 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:59.069929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26a748-5758-4633-b72e-9bc56f199281 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:59.070207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26a748-5758-4633-b72e-9bc56f199281 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:59.070985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26a748-5758-4633-b72e-9bc56f199281 None admin] RBAC: Adding query filter params (name=member) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:39:59.071273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26a748-5758-4633-b72e-9bc56f199281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:59.072863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26a748-5758-4633-b72e-9bc56f199281 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:59.079340 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d21928-be31-4fad-bc28-a1ab3861ecbf None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:59.082787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 101/215] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:39:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:59.089411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26a748-5758-4633-b72e-9bc56f199281 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:59.118616 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2baffd9-f37f-43f0-916b-79b7268b1150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:59.119680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baffd9-f37f-43f0-916b-79b7268b1150 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:59.120162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baffd9-f37f-43f0-916b-79b7268b1150 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:59.120666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2baffd9-f37f-43f0-916b-79b7268b1150 None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:59.121603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2baffd9-f37f-43f0-916b-79b7268b1150 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:59.123294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2baffd9-f37f-43f0-916b-79b7268b1150 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:59.159982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 115/216] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:39:58 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:39:59.192245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:59.227714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26a748-5758-4633-b72e-9bc56f199281 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:59.274520 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786667fa-b754-42cf-8298-53b410f676ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:59.275426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786667fa-b754-42cf-8298-53b410f676ce None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:59.275705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786667fa-b754-42cf-8298-53b410f676ce None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:59.275978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786667fa-b754-42cf-8298-53b410f676ce None admin] PATH_INFO: `/v3/users/f065445e580f4df996944746ea5b32f8` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:59.293905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786667fa-b754-42cf-8298-53b410f676ce None admin] RBAC: Authorizing `identity:get_user(user_id=f065445e580f4df996944746ea5b32f8)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:59.295469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786667fa-b754-42cf-8298-53b410f676ce None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:59.297730 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 102/217] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:39:59 2022] GET /identity/v3/users/f065445e580f4df996944746ea5b32f8 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:39:59.303655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2baffd9-f37f-43f0-916b-79b7268b1150 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:59.370144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:59.423642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2baffd9-f37f-43f0-916b-79b7268b1150 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:59.480139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f377fca-ae58-4656-badc-619dfa7187b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:59.480731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f377fca-ae58-4656-badc-619dfa7187b5 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:59.481018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f377fca-ae58-4656-badc-619dfa7187b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:59.481292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f377fca-ae58-4656-badc-619dfa7187b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:59.556961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f377fca-ae58-4656-badc-619dfa7187b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:59.558443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f377fca-ae58-4656-badc-619dfa7187b5 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:59.620747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:59.674228 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 116/218] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:39:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:39:59.681154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786667fa-b754-42cf-8298-53b410f676ce None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:59.813740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786667fa-b754-42cf-8298-53b410f676ce None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:39:59.855298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:39:59.866666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a0823-8843-432e-b2c2-d24460b83d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:39:59.867562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a0823-8843-432e-b2c2-d24460b83d31 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:59.867853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a0823-8843-432e-b2c2-d24460b83d31 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:59.868132 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a0823-8843-432e-b2c2-d24460b83d31 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:59.869406 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a0823-8843-432e-b2c2-d24460b83d31 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:39:59.870864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a0823-8843-432e-b2c2-d24460b83d31 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:39:59.873946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 103/219] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:39:59 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:39:59.879170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f377fca-ae58-4656-badc-619dfa7187b5 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:39:59.971033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c6ac9-b0ef-4550-9a72-9da9f4903943 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:59.971526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c6ac9-b0ef-4550-9a72-9da9f4903943 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:59.972000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c6ac9-b0ef-4550-9a72-9da9f4903943 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:39:59.975049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 104/220] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:39:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:39:59.981305 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae8d4e-26c5-49e3-ab31-a290ddb82e15 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:39:59.981744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae8d4e-26c5-49e3-ab31-a290ddb82e15 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:39:59.982202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae8d4e-26c5-49e3-ab31-a290ddb82e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:00.014782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-dfae8d4e-26c5-49e3-ab31-a290ddb82e15 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:00.016592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-dfae8d4e-26c5-49e3-ab31-a290ddb82e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:00.073209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f377fca-ae58-4656-badc-619dfa7187b5 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:00.122674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 105/221] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:40:00.124086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fbe8f-5ed6-4d43-8f8a-5b1d5b372ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:00.124546 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03475f1-da01-4fc5-8636-8203c39143b8 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:00.124917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fbe8f-5ed6-4d43-8f8a-5b1d5b372ec0 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:00.125022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03475f1-da01-4fc5-8636-8203c39143b8 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:00.125318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03475f1-da01-4fc5-8636-8203c39143b8 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:00.125429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fbe8f-5ed6-4d43-8f8a-5b1d5b372ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:00.125786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fbe8f-5ed6-4d43-8f8a-5b1d5b372ec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:00.126726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fbe8f-5ed6-4d43-8f8a-5b1d5b372ec0 None admin] RBAC: Adding query filter params (role.id=0f8e73b285204863a74f8af4ff7bbc86, scope.project.id=4060784cbaea460b93f362ae4cf23508, user.id=f065445e580f4df996944746ea5b32f8) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:00.127159 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fbe8f-5ed6-4d43-8f8a-5b1d5b372ec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:00.127620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 106/222] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:00.129200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fbe8f-5ed6-4d43-8f8a-5b1d5b372ec0 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:00.139079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37f1a0-56f9-4518-adf1-ea66ea57150b None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:00.139392 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37f1a0-56f9-4518-adf1-ea66ea57150b None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:00.140471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37f1a0-56f9-4518-adf1-ea66ea57150b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:00.144030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 117/223] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:39:59 2022] GET /identity/v3/role_assignments?user.id=f065445e580f4df996944746ea5b32f8&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:00.156827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fbe8f-5ed6-4d43-8f8a-5b1d5b372ec0 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:00.174848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ee37f1a0-56f9-4518-adf1-ea66ea57150b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:00.176687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ee37f1a0-56f9-4518-adf1-ea66ea57150b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:00.195130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:00.366221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 107/224] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:00.368656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fbe8f-5ed6-4d43-8f8a-5b1d5b372ec0 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:00.404932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37f1a0-56f9-4518-adf1-ea66ea57150b None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:00.428207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ec8ac-f818-4f2b-a63d-27a5cb6b9ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:00.429123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec8ac-f818-4f2b-a63d-27a5cb6b9ed6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:00.429407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec8ac-f818-4f2b-a63d-27a5cb6b9ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:00.429681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec8ac-f818-4f2b-a63d-27a5cb6b9ed6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:00.451330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ec8ac-f818-4f2b-a63d-27a5cb6b9ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-a75ec8ac-f818-4f2b-a63d-27a5cb6b9ed6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:00.461663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:00.462744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:00.463545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:00.466931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 118/225] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:40:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:00.476241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ec8ac-f818-4f2b-a63d-27a5cb6b9ed6 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:00.509500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:00.591922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37f1a0-56f9-4518-adf1-ea66ea57150b None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:00.606934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:00.637614 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863bf92-7082-47f0-b461-0e2a8321a60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:00.638338 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863bf92-7082-47f0-b461-0e2a8321a60d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:00.638624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863bf92-7082-47f0-b461-0e2a8321a60d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:00.638899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863bf92-7082-47f0-b461-0e2a8321a60d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:00.657821 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ec8ac-f818-4f2b-a63d-27a5cb6b9ed6 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:00.660219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863bf92-7082-47f0-b461-0e2a8321a60d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-4863bf92-7082-47f0-b461-0e2a8321a60d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:00.674792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:00.675760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:00.676853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 108/226] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:40:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:00.685240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863bf92-7082-47f0-b461-0e2a8321a60d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:00.724027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d1e94-730d-4d89-aa26-31947e9233e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:00.725136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d1e94-730d-4d89-aa26-31947e9233e8 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:00.725559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d1e94-730d-4d89-aa26-31947e9233e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:00.725978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d1e94-730d-4d89-aa26-31947e9233e8 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:00.727177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d1e94-730d-4d89-aa26-31947e9233e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:00.727696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d1e94-730d-4d89-aa26-31947e9233e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:00.730418 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d1e94-730d-4d89-aa26-31947e9233e8 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:00.742764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 119/227] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:00.748857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d1e94-730d-4d89-aa26-31947e9233e8 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:00.908458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863bf92-7082-47f0-b461-0e2a8321a60d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:00.929434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:00.946628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e82d96-de40-4bba-b070-c16e7a26f9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:00.947320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82d96-de40-4bba-b070-c16e7a26f9cf None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:00.947603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82d96-de40-4bba-b070-c16e7a26f9cf None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:00.947882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82d96-de40-4bba-b070-c16e7a26f9cf None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:00.948795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e82d96-de40-4bba-b070-c16e7a26f9cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:00.949676 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e82d96-de40-4bba-b070-c16e7a26f9cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:00.950955 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e82d96-de40-4bba-b070-c16e7a26f9cf None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:00.953840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d1e94-730d-4d89-aa26-31947e9233e8 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:00.961965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 109/228] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:00.968939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e82d96-de40-4bba-b070-c16e7a26f9cf None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:01.037357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8b870-23cd-45aa-ba4a-c26945b6bb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:01.039103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8b870-23cd-45aa-ba4a-c26945b6bb9b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:01.039603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8b870-23cd-45aa-ba4a-c26945b6bb9b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:01.040108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8b870-23cd-45aa-ba4a-c26945b6bb9b None admin] PATH_INFO: `/v3/users/6caffb9893c44b68a67f8c44e43db934` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:01.042602 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8b870-23cd-45aa-ba4a-c26945b6bb9b None admin] RBAC: Authorizing `identity:get_user(user_id=6caffb9893c44b68a67f8c44e43db934)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:01.044431 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8b870-23cd-45aa-ba4a-c26945b6bb9b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:01.049307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 120/229] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:00 2022] GET /identity/v3/users/6caffb9893c44b68a67f8c44e43db934 => generated 283 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:01.056704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8b870-23cd-45aa-ba4a-c26945b6bb9b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:01.096821 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:01.188387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e82d96-de40-4bba-b070-c16e7a26f9cf None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:01.238151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc57560-0846-41ba-92b3-03b05fb8204b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:01.239103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc57560-0846-41ba-92b3-03b05fb8204b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:01.239391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc57560-0846-41ba-92b3-03b05fb8204b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:01.239663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc57560-0846-41ba-92b3-03b05fb8204b None admin] PATH_INFO: `/v3/users/6caffb9893c44b68a67f8c44e43db934` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:01.242122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc57560-0846-41ba-92b3-03b05fb8204b None admin] RBAC: Authorizing `identity:get_user(user_id=6caffb9893c44b68a67f8c44e43db934)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:01.243586 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc57560-0846-41ba-92b3-03b05fb8204b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:01.245493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 110/230] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:00 2022] GET /identity/v3/users/6caffb9893c44b68a67f8c44e43db934 => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:01.251229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc57560-0846-41ba-92b3-03b05fb8204b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:01.277393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8b870-23cd-45aa-ba4a-c26945b6bb9b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:01.348270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:01.397625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311f779-fc39-47ef-99e8-df51f33991f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:01.398868 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311f779-fc39-47ef-99e8-df51f33991f9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:01.399393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311f779-fc39-47ef-99e8-df51f33991f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:01.399864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311f779-fc39-47ef-99e8-df51f33991f9 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:01.405174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311f779-fc39-47ef-99e8-df51f33991f9 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:01.407053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311f779-fc39-47ef-99e8-df51f33991f9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:01.409101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 121/231] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:01 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:01.414724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311f779-fc39-47ef-99e8-df51f33991f9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:01.419765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc57560-0846-41ba-92b3-03b05fb8204b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:01.481076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ca8d6-a1ac-48de-98e9-8e723f932564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:01.482155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ca8d6-a1ac-48de-98e9-8e723f932564 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:01.483584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ca8d6-a1ac-48de-98e9-8e723f932564 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:01.483867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ca8d6-a1ac-48de-98e9-8e723f932564 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:01.487700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ca8d6-a1ac-48de-98e9-8e723f932564 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:01.490749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ca8d6-a1ac-48de-98e9-8e723f932564 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:01.497306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 111/232] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:01 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:01.503028 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ca8d6-a1ac-48de-98e9-8e723f932564 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:01.604697 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:01.671180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311f779-fc39-47ef-99e8-df51f33991f9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:01.737133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27043b86-6c61-4a91-be08-6880f2efcd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:01.738496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27043b86-6c61-4a91-be08-6880f2efcd71 None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:01.738988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27043b86-6c61-4a91-be08-6880f2efcd71 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:01.739492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27043b86-6c61-4a91-be08-6880f2efcd71 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508/users/6caffb9893c44b68a67f8c44e43db934/roles/aab847245001401cb4d3d43238298f07` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:01.744989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27043b86-6c61-4a91-be08-6880f2efcd71 None admin] RBAC: Authorizing `identity:create_grant(project_id=4060784cbaea460b93f362ae4cf23508, user_id=6caffb9893c44b68a67f8c44e43db934, role_id=aab847245001401cb4d3d43238298f07)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:01.748207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27043b86-6c61-4a91-be08-6880f2efcd71 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:01.776582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ca8d6-a1ac-48de-98e9-8e723f932564 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:01.793170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 122/233] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:40:01 2022] PUT /identity/v3/projects/4060784cbaea460b93f362ae4cf23508/users/6caffb9893c44b68a67f8c44e43db934/roles/aab847245001401cb4d3d43238298f07 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:40:01.816790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:01.849252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bb8e9-33ce-47cb-a1da-cb5eb0190272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:01.851241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb8e9-33ce-47cb-a1da-cb5eb0190272 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:01.851935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb8e9-33ce-47cb-a1da-cb5eb0190272 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:01.852719 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb8e9-33ce-47cb-a1da-cb5eb0190272 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508/users/6caffb9893c44b68a67f8c44e43db934/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:01.856393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bb8e9-33ce-47cb-a1da-cb5eb0190272 None admin] RBAC: Authorizing `identity:create_grant(project_id=4060784cbaea460b93f362ae4cf23508, user_id=6caffb9893c44b68a67f8c44e43db934, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:01.859419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bb8e9-33ce-47cb-a1da-cb5eb0190272 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:01.886282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 112/234] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:40:01 2022] PUT /identity/v3/projects/4060784cbaea460b93f362ae4cf23508/users/6caffb9893c44b68a67f8c44e43db934/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:40:02.080268 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8473e-1a85-4523-9fee-6d588bdd87aa None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:02.080777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8473e-1a85-4523-9fee-6d588bdd87aa None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:02.081151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8473e-1a85-4523-9fee-6d588bdd87aa None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:02.083899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 123/235] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:02.093159 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2196f-a1f5-4bba-9c9e-dc9eb4eb8ddd None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:02.093631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2196f-a1f5-4bba-9c9e-dc9eb4eb8ddd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:02.094135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2196f-a1f5-4bba-9c9e-dc9eb4eb8ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:02.125685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-39e2196f-a1f5-4bba-9c9e-dc9eb4eb8ddd None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:02.127651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-39e2196f-a1f5-4bba-9c9e-dc9eb4eb8ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:02.242555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 113/236] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:02.283685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8473e-1a85-4523-9fee-6d588bdd87aa None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:02.315357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003b301-af6f-4188-8ce4-1e1ddbe91245 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:02.315770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003b301-af6f-4188-8ce4-1e1ddbe91245 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:02.316057 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003b301-af6f-4188-8ce4-1e1ddbe91245 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:02.317523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 114/237] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:02.326110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331d866-75a2-41fb-9793-1a67c227c427 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:02.326461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331d866-75a2-41fb-9793-1a67c227c427 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:02.326777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331d866-75a2-41fb-9793-1a67c227c427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:02.363879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e331d866-75a2-41fb-9793-1a67c227c427 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:02.370039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e331d866-75a2-41fb-9793-1a67c227c427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:02.517263 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 115/238] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 19 20:40:02.523110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8473e-1a85-4523-9fee-6d588bdd87aa None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:02.563180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331d866-75a2-41fb-9793-1a67c227c427 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:02.586250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3217e5-6f27-46be-808f-3cc99227f99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:02.587069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3217e5-6f27-46be-808f-3cc99227f99b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:02.587354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3217e5-6f27-46be-808f-3cc99227f99b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:02.587728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3217e5-6f27-46be-808f-3cc99227f99b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:02.610282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:02.620415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3217e5-6f27-46be-808f-3cc99227f99b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-0e3217e5-6f27-46be-808f-3cc99227f99b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:02.632196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:02.633848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:02.635245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 124/239] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:02.642959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3217e5-6f27-46be-808f-3cc99227f99b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:02.649975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:02.781108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331d866-75a2-41fb-9793-1a67c227c427 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:02.850803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f637e5-3557-46dc-858c-9796291a6947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:02.851620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f637e5-3557-46dc-858c-9796291a6947 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:02.851904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f637e5-3557-46dc-858c-9796291a6947 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:02.852180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f637e5-3557-46dc-858c-9796291a6947 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:02.853495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f637e5-3557-46dc-858c-9796291a6947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:02.854958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f637e5-3557-46dc-858c-9796291a6947 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:02.856962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 116/240] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:02.865848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f637e5-3557-46dc-858c-9796291a6947 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:02.899263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3217e5-6f27-46be-808f-3cc99227f99b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:02.942795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985961-ac75-40e3-9965-ab502c0c7c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:02.944047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985961-ac75-40e3-9965-ab502c0c7c71 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:02.944360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985961-ac75-40e3-9965-ab502c0c7c71 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:02.944634 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985961-ac75-40e3-9965-ab502c0c7c71 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:02.945396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985961-ac75-40e3-9965-ab502c0c7c71 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:02.945705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985961-ac75-40e3-9965-ab502c0c7c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:02.947109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985961-ac75-40e3-9965-ab502c0c7c71 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:02.967057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 125/241] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:02.972951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985961-ac75-40e3-9965-ab502c0c7c71 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:03.066721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:03.074843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f637e5-3557-46dc-858c-9796291a6947 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:03.085529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:03.116903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985961-ac75-40e3-9965-ab502c0c7c71 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:03.117263 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef865123-c919-4c88-b80e-f5b7e3d2fe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:03.118229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef865123-c919-4c88-b80e-f5b7e3d2fe6a None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:03.118874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef865123-c919-4c88-b80e-f5b7e3d2fe6a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:03.119360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef865123-c919-4c88-b80e-f5b7e3d2fe6a None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:03.120946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef865123-c919-4c88-b80e-f5b7e3d2fe6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:03.123662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef865123-c919-4c88-b80e-f5b7e3d2fe6a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:03.155486 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 117/242] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:40:02 2022] POST /identity/v3/users => generated 293 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:40:03.156757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046dfc9-aa1a-4fe3-af8d-ccc410fa3e10 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:03.157045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046dfc9-aa1a-4fe3-af8d-ccc410fa3e10 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:03.157324 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046dfc9-aa1a-4fe3-af8d-ccc410fa3e10 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:03.158581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 118/243] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:03.176357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438df3e-89c2-430f-b835-6abec808abb7 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:03.176357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438df3e-89c2-430f-b835-6abec808abb7 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:03.176357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438df3e-89c2-430f-b835-6abec808abb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:03.193015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faf131-d9c1-43a8-9459-357d36f669dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:03.195160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faf131-d9c1-43a8-9459-357d36f669dc None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:03.195662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faf131-d9c1-43a8-9459-357d36f669dc None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:03.196155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faf131-d9c1-43a8-9459-357d36f669dc None admin] PATH_INFO: `/v3/users/1a2c11e17dab480d9b4cd44f25df49ef` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:03.239545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c438df3e-89c2-430f-b835-6abec808abb7 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:03.241399 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c438df3e-89c2-430f-b835-6abec808abb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:03.247732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5faf131-d9c1-43a8-9459-357d36f669dc None admin] RBAC: Authorizing `identity:get_user(user_id=1a2c11e17dab480d9b4cd44f25df49ef)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:03.250396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5faf131-d9c1-43a8-9459-357d36f669dc None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:03.260094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 126/244] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:02 2022] GET /identity/v3/users/1a2c11e17dab480d9b4cd44f25df49ef => generated 297 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:03.261687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faf131-d9c1-43a8-9459-357d36f669dc None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:03.405159 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 119/245] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:03.432928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438df3e-89c2-430f-b835-6abec808abb7 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:03.456867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faf131-d9c1-43a8-9459-357d36f669dc None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:03.488241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:03.537471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cf819-6508-4eb8-b88a-a4b8ab655374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:03.538435 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf819-6508-4eb8-b88a-a4b8ab655374 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:03.538727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf819-6508-4eb8-b88a-a4b8ab655374 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:03.539006 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cf819-6508-4eb8-b88a-a4b8ab655374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:03.575688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:03.590044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438df3e-89c2-430f-b835-6abec808abb7 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:03.640831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499dbc8-5a12-4134-9801-e2c0d3592901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:03.641935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499dbc8-5a12-4134-9801-e2c0d3592901 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:03.642390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499dbc8-5a12-4134-9801-e2c0d3592901 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:03.642977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499dbc8-5a12-4134-9801-e2c0d3592901 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:03.654308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cf819-6508-4eb8-b88a-a4b8ab655374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:03.655566 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499dbc8-5a12-4134-9801-e2c0d3592901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:03.656095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cf819-6508-4eb8-b88a-a4b8ab655374 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-0499dbc8-5a12-4134-9801-e2c0d3592901 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:03.663904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:03.664937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:03.665752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 120/246] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:03.672610 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499dbc8-5a12-4134-9801-e2c0d3592901 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:03.790441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499dbc8-5a12-4134-9801-e2c0d3592901 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:03.843189 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242ef4b-6c7f-4a73-9612-fc900c39db8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:03.844235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242ef4b-6c7f-4a73-9612-fc900c39db8c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:03.845004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242ef4b-6c7f-4a73-9612-fc900c39db8c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:03.845717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242ef4b-6c7f-4a73-9612-fc900c39db8c None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:03.847453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242ef4b-6c7f-4a73-9612-fc900c39db8c None admin] RBAC: Adding query filter params (name=member) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:03.848231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242ef4b-6c7f-4a73-9612-fc900c39db8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:03.850092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242ef4b-6c7f-4a73-9612-fc900c39db8c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:03.853505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 127/247] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:03.863749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cf819-6508-4eb8-b88a-a4b8ab655374 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:03.884738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 121/248] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:03.891389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242ef4b-6c7f-4a73-9612-fc900c39db8c None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:03.928917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:03.940554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:04.206249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cf819-6508-4eb8-b88a-a4b8ab655374 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:04.238366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242ef4b-6c7f-4a73-9612-fc900c39db8c None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:04.316051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2844df8-ae04-49b9-a8a2-f7f0e57cadf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:04.321337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2844df8-ae04-49b9-a8a2-f7f0e57cadf2 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:04.321627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2844df8-ae04-49b9-a8a2-f7f0e57cadf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:04.321906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2844df8-ae04-49b9-a8a2-f7f0e57cadf2 None admin] PATH_INFO: `/v3/users/f065445e580f4df996944746ea5b32f8` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:04.323208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2844df8-ae04-49b9-a8a2-f7f0e57cadf2 None admin] RBAC: Authorizing `identity:get_user(user_id=f065445e580f4df996944746ea5b32f8)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:04.325002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2844df8-ae04-49b9-a8a2-f7f0e57cadf2 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:04.327092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 122/249] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:03 2022] GET /identity/v3/users/f065445e580f4df996944746ea5b32f8 => generated 297 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:04.329390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbbb3f-48fd-4488-a50f-9412983dc8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:04.330289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbbb3f-48fd-4488-a50f-9412983dc8cd None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:04.330572 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbbb3f-48fd-4488-a50f-9412983dc8cd None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:04.330849 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbbb3f-48fd-4488-a50f-9412983dc8cd None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:04.332192 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbbb3f-48fd-4488-a50f-9412983dc8cd None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:04.333755 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbbb3f-48fd-4488-a50f-9412983dc8cd None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:04.335835 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 128/250] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:03 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:04.341779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbbb3f-48fd-4488-a50f-9412983dc8cd None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:04.342339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2844df8-ae04-49b9-a8a2-f7f0e57cadf2 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:04.581999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:04.598006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:04.620157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbbb3f-48fd-4488-a50f-9412983dc8cd None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:04.643920 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2844df8-ae04-49b9-a8a2-f7f0e57cadf2 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:04.690382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bfa23-d878-4aba-aea2-563a51bee32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:04.690990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bfa23-d878-4aba-aea2-563a51bee32f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:04.691274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bfa23-d878-4aba-aea2-563a51bee32f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:04.691551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bfa23-d878-4aba-aea2-563a51bee32f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:04.692458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bfa23-d878-4aba-aea2-563a51bee32f None admin] RBAC: Adding query filter params (role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2, scope.project.id=4060784cbaea460b93f362ae4cf23508, user.id=1a2c11e17dab480d9b4cd44f25df49ef) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:04.692776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bfa23-d878-4aba-aea2-563a51bee32f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:04.694862 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bfa23-d878-4aba-aea2-563a51bee32f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:04.710766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 129/251] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:04 2022] GET /identity/v3/role_assignments?user.id=1a2c11e17dab480d9b4cd44f25df49ef&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2 => generated 259 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:04.765698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6b902-0ce2-4e73-a879-0621f899b722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:04.767741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6b902-0ce2-4e73-a879-0621f899b722 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:04.768596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6b902-0ce2-4e73-a879-0621f899b722 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:04.769378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6b902-0ce2-4e73-a879-0621f899b722 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:04.772582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6b902-0ce2-4e73-a879-0621f899b722 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:04.775766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6b902-0ce2-4e73-a879-0621f899b722 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:04.779096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 123/252] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:04 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:04.789080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bfa23-d878-4aba-aea2-563a51bee32f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:04.987765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee23c71-cc0c-424b-ba06-fcbe025dabfc None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:04.988812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee23c71-cc0c-424b-ba06-fcbe025dabfc None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:04.989592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee23c71-cc0c-424b-ba06-fcbe025dabfc None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:04.992611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 124/253] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:05.003021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48db3e-5080-4bc8-93af-fd3a72b91de9 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:05.003956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48db3e-5080-4bc8-93af-fd3a72b91de9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:05.004835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48db3e-5080-4bc8-93af-fd3a72b91de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:05.042121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-fb48db3e-5080-4bc8-93af-fd3a72b91de9 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:05.046902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bfa23-d878-4aba-aea2-563a51bee32f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:05.049811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fb48db3e-5080-4bc8-93af-fd3a72b91de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:05.117490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf79330-4439-406c-b82e-3fb51dc782a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:05.119452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf79330-4439-406c-b82e-3fb51dc782a7 None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:05.120115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf79330-4439-406c-b82e-3fb51dc782a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:05.121763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf79330-4439-406c-b82e-3fb51dc782a7 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508/users/f065445e580f4df996944746ea5b32f8/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:05.125348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf79330-4439-406c-b82e-3fb51dc782a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4060784cbaea460b93f362ae4cf23508, user_id=f065445e580f4df996944746ea5b32f8, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:05.128474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf79330-4439-406c-b82e-3fb51dc782a7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:05.155938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 130/254] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:40:04 2022] PUT /identity/v3/projects/4060784cbaea460b93f362ae4cf23508/users/f065445e580f4df996944746ea5b32f8/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:40:05.181747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 125/255] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:05.218081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf79330-4439-406c-b82e-3fb51dc782a7 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:05.235972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:05.362273 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf79330-4439-406c-b82e-3fb51dc782a7 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:05.414311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2b811-1f79-4370-80e3-87814f87b187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:05.415159 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b811-1f79-4370-80e3-87814f87b187 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:05.415519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b811-1f79-4370-80e3-87814f87b187 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:05.415819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b811-1f79-4370-80e3-87814f87b187 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:05.433296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2b811-1f79-4370-80e3-87814f87b187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-11d2b811-1f79-4370-80e3-87814f87b187 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:05.441800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:05.442692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:05.443613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 131/256] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:40:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:05.450662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48db3e-5080-4bc8-93af-fd3a72b91de9 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:05.529600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdabf5-37f3-49d0-9e7c-1a157e01eaba None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:05.529927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdabf5-37f3-49d0-9e7c-1a157e01eaba None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:05.530192 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdabf5-37f3-49d0-9e7c-1a157e01eaba None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:05.531870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 132/257] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:05.539271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783cbd0-cfd4-4269-b524-bf5c4c9238ee None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:05.539566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783cbd0-cfd4-4269-b524-bf5c4c9238ee None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:05.540357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783cbd0-cfd4-4269-b524-bf5c4c9238ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:05.561829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-8783cbd0-cfd4-4269-b524-bf5c4c9238ee None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:05.563578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8783cbd0-cfd4-4269-b524-bf5c4c9238ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:05.567647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:05.595290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48db3e-5080-4bc8-93af-fd3a72b91de9 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:05.644145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:05.650784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 133/258] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:40:05.654064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f8e1e-d75a-43a3-8654-da612fea4034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:05.655328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f8e1e-d75a-43a3-8654-da612fea4034 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:05.655834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f8e1e-d75a-43a3-8654-da612fea4034 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:05.656350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f8e1e-d75a-43a3-8654-da612fea4034 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:05.657312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f8e1e-d75a-43a3-8654-da612fea4034 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:05.657821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f8e1e-d75a-43a3-8654-da612fea4034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:05.659422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f8e1e-d75a-43a3-8654-da612fea4034 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:05.675506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 126/259] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:05.677477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783cbd0-cfd4-4269-b524-bf5c4c9238ee None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:05.681952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f8e1e-d75a-43a3-8654-da612fea4034 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:05.813730 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f8e1e-d75a-43a3-8654-da612fea4034 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:05.869218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83da8c9-9be0-4d01-bbc3-8ff8d8d06b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:05.870113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83da8c9-9be0-4d01-bbc3-8ff8d8d06b1b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:05.870400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83da8c9-9be0-4d01-bbc3-8ff8d8d06b1b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:05.870675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83da8c9-9be0-4d01-bbc3-8ff8d8d06b1b None admin] PATH_INFO: `/v3/users/6caffb9893c44b68a67f8c44e43db934` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:05.872548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83da8c9-9be0-4d01-bbc3-8ff8d8d06b1b None admin] RBAC: Authorizing `identity:get_user(user_id=6caffb9893c44b68a67f8c44e43db934)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:05.874005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83da8c9-9be0-4d01-bbc3-8ff8d8d06b1b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:05.877523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 127/260] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:05 2022] GET /identity/v3/users/6caffb9893c44b68a67f8c44e43db934 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:05.882036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83da8c9-9be0-4d01-bbc3-8ff8d8d06b1b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:05.894668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783cbd0-cfd4-4269-b524-bf5c4c9238ee None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:05.905435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:05.990069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08b54d-bb38-45e4-acbd-97c065b4cb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:05.991256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08b54d-bb38-45e4-acbd-97c065b4cb78 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:05.991772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08b54d-bb38-45e4-acbd-97c065b4cb78 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:05.992245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08b54d-bb38-45e4-acbd-97c065b4cb78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:06.000270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83da8c9-9be0-4d01-bbc3-8ff8d8d06b1b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:06.032971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb08b54d-bb38-45e4-acbd-97c065b4cb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:06.036942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8cef4-7686-4267-b1e2-eb7c54e0116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:06.037739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8cef4-7686-4267-b1e2-eb7c54e0116a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:06.038131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8cef4-7686-4267-b1e2-eb7c54e0116a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:06.038515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8cef4-7686-4267-b1e2-eb7c54e0116a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-eb08b54d-bb38-45e4-acbd-97c065b4cb78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:06.042287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:06.043191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:06.044177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:06.046285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 134/261] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:40:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:06.069417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08b54d-bb38-45e4-acbd-97c065b4cb78 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:06.122022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8cef4-7686-4267-b1e2-eb7c54e0116a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:06.128350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8cef4-7686-4267-b1e2-eb7c54e0116a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:06.144807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:06.187935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:06.244197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08b54d-bb38-45e4-acbd-97c065b4cb78 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:06.276389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 128/262] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:06.282257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8cef4-7686-4267-b1e2-eb7c54e0116a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:06.313654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da220adf-f4ff-42ad-8fb5-2c46f25210e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:06.314621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da220adf-f4ff-42ad-8fb5-2c46f25210e9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:06.315048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da220adf-f4ff-42ad-8fb5-2c46f25210e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:06.315452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da220adf-f4ff-42ad-8fb5-2c46f25210e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:06.316366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da220adf-f4ff-42ad-8fb5-2c46f25210e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:06.316862 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da220adf-f4ff-42ad-8fb5-2c46f25210e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:06.319102 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da220adf-f4ff-42ad-8fb5-2c46f25210e9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:06.333369 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 135/263] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:06.349894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da220adf-f4ff-42ad-8fb5-2c46f25210e9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:06.539401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da220adf-f4ff-42ad-8fb5-2c46f25210e9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:06.562825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8cef4-7686-4267-b1e2-eb7c54e0116a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:06.580641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:06.587281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:06.614914 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9966124-5b77-491e-acb6-82db0c2ee384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:06.616342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9966124-5b77-491e-acb6-82db0c2ee384 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:06.616629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9966124-5b77-491e-acb6-82db0c2ee384 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:06.616938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9966124-5b77-491e-acb6-82db0c2ee384 None admin] PATH_INFO: `/v3/users/6caffb9893c44b68a67f8c44e43db934` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:06.621275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9966124-5b77-491e-acb6-82db0c2ee384 None admin] RBAC: Authorizing `identity:get_user(user_id=6caffb9893c44b68a67f8c44e43db934)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:06.622998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9966124-5b77-491e-acb6-82db0c2ee384 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:06.624975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 136/264] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:06 2022] GET /identity/v3/users/6caffb9893c44b68a67f8c44e43db934 => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:06.629875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337f893-5dcc-4533-b69a-3314e30754f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:06.630295 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9966124-5b77-491e-acb6-82db0c2ee384 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:06.630865 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337f893-5dcc-4533-b69a-3314e30754f1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:06.631191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337f893-5dcc-4533-b69a-3314e30754f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:06.631515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337f893-5dcc-4533-b69a-3314e30754f1 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:06.632809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337f893-5dcc-4533-b69a-3314e30754f1 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:06.634292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337f893-5dcc-4533-b69a-3314e30754f1 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:06.636067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 129/265] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:06 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:06.642313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337f893-5dcc-4533-b69a-3314e30754f1 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:06.804888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9966124-5b77-491e-acb6-82db0c2ee384 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:06.850667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d2cc3-5b67-4225-b9ce-4c447945485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:06.851702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d2cc3-5b67-4225-b9ce-4c447945485c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:06.852275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d2cc3-5b67-4225-b9ce-4c447945485c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:06.852798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d2cc3-5b67-4225-b9ce-4c447945485c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:06.867048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337f893-5dcc-4533-b69a-3314e30754f1 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:06.942791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686fd70-e8e0-4383-a75d-c498d7d9ab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:06.943492 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686fd70-e8e0-4383-a75d-c498d7d9ab78 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:06.943781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686fd70-e8e0-4383-a75d-c498d7d9ab78 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:06.944059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686fd70-e8e0-4383-a75d-c498d7d9ab78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:06.944922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686fd70-e8e0-4383-a75d-c498d7d9ab78 None admin] RBAC: Adding query filter params (role.id=aab847245001401cb4d3d43238298f07, scope.project.id=4060784cbaea460b93f362ae4cf23508, user.id=6caffb9893c44b68a67f8c44e43db934) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:06.945235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686fd70-e8e0-4383-a75d-c498d7d9ab78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:06.946690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686fd70-e8e0-4383-a75d-c498d7d9ab78 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:06.963122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 130/266] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:06 2022] GET /identity/v3/role_assignments?user.id=6caffb9893c44b68a67f8c44e43db934&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=aab847245001401cb4d3d43238298f07 => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:06.966077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d4f0-d026-43b8-b512-d0f293b4f3d9 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:06.966382 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d4f0-d026-43b8-b512-d0f293b4f3d9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:06.966661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578d4f0-d026-43b8-b512-d0f293b4f3d9 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:06.967657 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:06.970498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 131/267] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:06.975469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48832bae-ce6b-450c-b9d9-8d4289bed769 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:06.975760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48832bae-ce6b-450c-b9d9-8d4289bed769 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:06.976041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48832bae-ce6b-450c-b9d9-8d4289bed769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:06.987612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d2cc3-5b67-4225-b9ce-4c447945485c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:06.989116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d2cc3-5b67-4225-b9ce-4c447945485c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:07.026780 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-48832bae-ce6b-450c-b9d9-8d4289bed769 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:07.028050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-48832bae-ce6b-450c-b9d9-8d4289bed769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:07.121714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:07.190461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 132/268] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:07.197869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66163f2c-cf6c-4734-8fb7-38d447cd2d8c None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:07.198186 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66163f2c-cf6c-4734-8fb7-38d447cd2d8c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:07.198435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66163f2c-cf6c-4734-8fb7-38d447cd2d8c None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:07.200224 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 133/269] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:07.218075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098c58-ac4b-4de4-87fd-b10d6e1db342 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:07.218368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098c58-ac4b-4de4-87fd-b10d6e1db342 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:07.218654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b098c58-ac4b-4de4-87fd-b10d6e1db342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:07.232678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 137/270] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:40:07.233548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d2cc3-5b67-4225-b9ce-4c447945485c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:07.247677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-3b098c58-ac4b-4de4-87fd-b10d6e1db342 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:07.248923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3b098c58-ac4b-4de4-87fd-b10d6e1db342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:07.321509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 134/271] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:40:07.329407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b098c58-ac4b-4de4-87fd-b10d6e1db342 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:07.367285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:07.381618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d2cc3-5b67-4225-b9ce-4c447945485c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:07.441133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16f5ca-5619-43c1-9497-5dc85aea53f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:07.442003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16f5ca-5619-43c1-9497-5dc85aea53f2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:07.442290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16f5ca-5619-43c1-9497-5dc85aea53f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:07.442567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16f5ca-5619-43c1-9497-5dc85aea53f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:07.460613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16f5ca-5619-43c1-9497-5dc85aea53f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-ce16f5ca-5619-43c1-9497-5dc85aea53f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:07.471220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:07.473660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 138/272] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:07.475184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16f5ca-5619-43c1-9497-5dc85aea53f2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:07.493469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b098c58-ac4b-4de4-87fd-b10d6e1db342 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:07.495330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:07.553574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17662c-aeb9-4450-808f-90733cbc0344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:07.554704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17662c-aeb9-4450-808f-90733cbc0344 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:07.555018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17662c-aeb9-4450-808f-90733cbc0344 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:07.555687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17662c-aeb9-4450-808f-90733cbc0344 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:07.557472 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17662c-aeb9-4450-808f-90733cbc0344 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:07.560112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17662c-aeb9-4450-808f-90733cbc0344 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:07.563086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 135/273] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:07 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:07.563976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17662c-aeb9-4450-808f-90733cbc0344 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:07.600636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16f5ca-5619-43c1-9497-5dc85aea53f2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:07.641483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecc772-7b2f-407c-9997-8285ac682fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:07.642402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecc772-7b2f-407c-9997-8285ac682fcc None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:07.642876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecc772-7b2f-407c-9997-8285ac682fcc None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:07.643341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ecc772-7b2f-407c-9997-8285ac682fcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:07.644829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ecc772-7b2f-407c-9997-8285ac682fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:07.647114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ecc772-7b2f-407c-9997-8285ac682fcc None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:07.649127 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 139/274] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:07.650176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecc772-7b2f-407c-9997-8285ac682fcc None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:07.679519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:07.712016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17662c-aeb9-4450-808f-90733cbc0344 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:07.757406 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a38ba1-39ac-4590-a542-6a421e89fcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:07.758356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a38ba1-39ac-4590-a542-6a421e89fcf8 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:07.758791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a38ba1-39ac-4590-a542-6a421e89fcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:07.759217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a38ba1-39ac-4590-a542-6a421e89fcf8 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:07.760126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a38ba1-39ac-4590-a542-6a421e89fcf8 None admin] RBAC: Adding query filter params (name=member) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:07.760614 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a38ba1-39ac-4590-a542-6a421e89fcf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:07.762258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a38ba1-39ac-4590-a542-6a421e89fcf8 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:07.773881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 136/275] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:07.775119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a38ba1-39ac-4590-a542-6a421e89fcf8 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:07.788005 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ecc772-7b2f-407c-9997-8285ac682fcc None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:07.826253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:07.830592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcaacc6-5f39-4fdd-b39a-6e491ec87e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:07.832092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcaacc6-5f39-4fdd-b39a-6e491ec87e1c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:07.832415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcaacc6-5f39-4fdd-b39a-6e491ec87e1c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:07.832707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcaacc6-5f39-4fdd-b39a-6e491ec87e1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:07.833564 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcaacc6-5f39-4fdd-b39a-6e491ec87e1c None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=4060784cbaea460b93f362ae4cf23508, user.id=6caffb9893c44b68a67f8c44e43db934) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:07.833937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcaacc6-5f39-4fdd-b39a-6e491ec87e1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:07.835403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcaacc6-5f39-4fdd-b39a-6e491ec87e1c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:07.843017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 140/276] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:07 2022] GET /identity/v3/role_assignments?user.id=6caffb9893c44b68a67f8c44e43db934&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:07.843947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcaacc6-5f39-4fdd-b39a-6e491ec87e1c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:07.918787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a38ba1-39ac-4590-a542-6a421e89fcf8 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:07.961564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b632d8-6a60-40b8-97bb-b590ddbaeffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:07.962647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b632d8-6a60-40b8-97bb-b590ddbaeffa None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:07.962871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:07.963207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b632d8-6a60-40b8-97bb-b590ddbaeffa None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:07.963808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b632d8-6a60-40b8-97bb-b590ddbaeffa None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:07.965045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b632d8-6a60-40b8-97bb-b590ddbaeffa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:07.968674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b632d8-6a60-40b8-97bb-b590ddbaeffa None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:07.991202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcaacc6-5f39-4fdd-b39a-6e491ec87e1c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:07.992109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 137/277] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:40:07 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:40:07.993325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa751d8-6062-4499-a15e-9dab478637c6 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:07.993613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa751d8-6062-4499-a15e-9dab478637c6 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:07.994106 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa751d8-6062-4499-a15e-9dab478637c6 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:07.995207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 138/278] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:08.000849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc730f-3d5c-44fb-96ba-7bf9dad13cb2 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:08.001155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc730f-3d5c-44fb-96ba-7bf9dad13cb2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:08.001567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc730f-3d5c-44fb-96ba-7bf9dad13cb2 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:08.003168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 139/279] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:08.007759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7483b-4358-4657-9254-bbbde5918852 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:08.007759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7483b-4358-4657-9254-bbbde5918852 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:08.007759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7483b-4358-4657-9254-bbbde5918852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:08.031365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-b3c7483b-4358-4657-9254-bbbde5918852 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:08.034906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b3c7483b-4358-4657-9254-bbbde5918852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:08.061457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271538b1-665e-44ab-a59a-f525cb323fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:08.062390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271538b1-665e-44ab-a59a-f525cb323fcb None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:08.063369 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271538b1-665e-44ab-a59a-f525cb323fcb None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:08.063655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271538b1-665e-44ab-a59a-f525cb323fcb None admin] PATH_INFO: `/v3/users/223ab3353b9e420f882f33a055acccb6` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:08.085734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271538b1-665e-44ab-a59a-f525cb323fcb None admin] RBAC: Authorizing `identity:get_user(user_id=223ab3353b9e420f882f33a055acccb6)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:08.087640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271538b1-665e-44ab-a59a-f525cb323fcb None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:08.091037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 141/280] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:07 2022] GET /identity/v3/users/223ab3353b9e420f882f33a055acccb6 => generated 293 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:08.094025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bb6cb-4fec-4533-997c-25f7e662b819 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:08.094538 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bb6cb-4fec-4533-997c-25f7e662b819 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:08.095033 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bb6cb-4fec-4533-997c-25f7e662b819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:08.119187 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4c1bb6cb-4fec-4533-997c-25f7e662b819 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:08.120774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4c1bb6cb-4fec-4533-997c-25f7e662b819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:08.135257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 140/281] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:08.136056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7483b-4358-4657-9254-bbbde5918852 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:08.173889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:08.210919 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 142/282] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:08.220351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bb6cb-4fec-4533-997c-25f7e662b819 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:08.281072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7483b-4358-4657-9254-bbbde5918852 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:08.313166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:08.327647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a990645-837f-402e-9afc-4c8a617a2c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:08.328752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a990645-837f-402e-9afc-4c8a617a2c12 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:08.329067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a990645-837f-402e-9afc-4c8a617a2c12 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:08.329483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a990645-837f-402e-9afc-4c8a617a2c12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:08.330462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a990645-837f-402e-9afc-4c8a617a2c12 None admin] RBAC: Adding query filter params (role.id=0f8e73b285204863a74f8af4ff7bbc86, user.id=223ab3353b9e420f882f33a055acccb6, scope.system=all) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:08.330780 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a990645-837f-402e-9afc-4c8a617a2c12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:08.332752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a990645-837f-402e-9afc-4c8a617a2c12 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:08.340977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 141/283] 10.222.0.85 () {56 vars in 1308 bytes} [Tue Jul 19 20:40:08 2022] GET /identity/v3/role_assignments?user.id=223ab3353b9e420f882f33a055acccb6&scope.system=all&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:08.341838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a990645-837f-402e-9afc-4c8a617a2c12 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:08.366861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bb6cb-4fec-4533-997c-25f7e662b819 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:08.432669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f666a3e-0995-4f9e-9727-0903b88aefe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:08.434415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f666a3e-0995-4f9e-9727-0903b88aefe2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:08.435145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f666a3e-0995-4f9e-9727-0903b88aefe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:08.435854 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f666a3e-0995-4f9e-9727-0903b88aefe2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:08.456633 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f666a3e-0995-4f9e-9727-0903b88aefe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-8f666a3e-0995-4f9e-9727-0903b88aefe2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:08.471887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:08.473485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:08.474916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:08.479136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 143/284] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:08.491865 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:08.497838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f666a3e-0995-4f9e-9727-0903b88aefe2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:08.519888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a990645-837f-402e-9afc-4c8a617a2c12 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:08.567201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde83c8-547b-4cc1-b8c8-a60975088eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:08.568073 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde83c8-547b-4cc1-b8c8-a60975088eef None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:08.569208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde83c8-547b-4cc1-b8c8-a60975088eef None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:08.569653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde83c8-547b-4cc1-b8c8-a60975088eef None admin] PATH_INFO: `/v3/roles/member` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:08.581674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde83c8-547b-4cc1-b8c8-a60975088eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-fbde83c8-547b-4cc1-b8c8-a60975088eef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:08.590158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:08.591108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:08.591912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 142/285] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:08.597603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde83c8-547b-4cc1-b8c8-a60975088eef None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:08.635540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f666a3e-0995-4f9e-9727-0903b88aefe2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:08.670425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:08.677993 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74db7c1-58e4-4641-aa44-5fbab9a2c6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:08.678724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74db7c1-58e4-4641-aa44-5fbab9a2c6a5 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:08.679000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74db7c1-58e4-4641-aa44-5fbab9a2c6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:08.679269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74db7c1-58e4-4641-aa44-5fbab9a2c6a5 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:08.680023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74db7c1-58e4-4641-aa44-5fbab9a2c6a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:08.680404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74db7c1-58e4-4641-aa44-5fbab9a2c6a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:08.682187 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74db7c1-58e4-4641-aa44-5fbab9a2c6a5 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:08.692062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 144/286] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:08.698098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74db7c1-58e4-4641-aa44-5fbab9a2c6a5 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:08.746555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde83c8-547b-4cc1-b8c8-a60975088eef None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:08.788953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f32b5e-ce39-4297-a9a4-fbff481ff3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:08.790121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f32b5e-ce39-4297-a9a4-fbff481ff3d4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:08.790611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f32b5e-ce39-4297-a9a4-fbff481ff3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:08.791358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f32b5e-ce39-4297-a9a4-fbff481ff3d4 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:08.792382 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f32b5e-ce39-4297-a9a4-fbff481ff3d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:08.792922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f32b5e-ce39-4297-a9a4-fbff481ff3d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:08.794564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f32b5e-ce39-4297-a9a4-fbff481ff3d4 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:08.806272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 143/287] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:08.812796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f32b5e-ce39-4297-a9a4-fbff481ff3d4 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:08.817324 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:08.944444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f32b5e-ce39-4297-a9a4-fbff481ff3d4 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:08.964721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74db7c1-58e4-4641-aa44-5fbab9a2c6a5 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:08.983628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c297503-33df-4c61-a0c5-fd12a6b2942f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:08.984775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c297503-33df-4c61-a0c5-fd12a6b2942f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:08.985138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c297503-33df-4c61-a0c5-fd12a6b2942f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:08.985489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c297503-33df-4c61-a0c5-fd12a6b2942f None admin] PATH_INFO: `/v3/users/f065445e580f4df996944746ea5b32f8` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:08.986916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c297503-33df-4c61-a0c5-fd12a6b2942f None admin] RBAC: Authorizing `identity:get_user(user_id=f065445e580f4df996944746ea5b32f8)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:08.992362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c297503-33df-4c61-a0c5-fd12a6b2942f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:08.992362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 144/288] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:08 2022] GET /identity/v3/users/f065445e580f4df996944746ea5b32f8 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:09.000752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c297503-33df-4c61-a0c5-fd12a6b2942f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:09.045348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085476f-88b3-42ce-8b0d-571c1f9d0402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:09.046296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085476f-88b3-42ce-8b0d-571c1f9d0402 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:09.046585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085476f-88b3-42ce-8b0d-571c1f9d0402 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:09.046865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085476f-88b3-42ce-8b0d-571c1f9d0402 None admin] PATH_INFO: `/v3/users/1a2c11e17dab480d9b4cd44f25df49ef` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:09.049792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085476f-88b3-42ce-8b0d-571c1f9d0402 None admin] RBAC: Authorizing `identity:get_user(user_id=1a2c11e17dab480d9b4cd44f25df49ef)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:09.051278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085476f-88b3-42ce-8b0d-571c1f9d0402 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:09.053715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 145/289] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:08 2022] GET /identity/v3/users/1a2c11e17dab480d9b4cd44f25df49ef => generated 297 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:09.059829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085476f-88b3-42ce-8b0d-571c1f9d0402 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:09.120490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:09.150054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c297503-33df-4c61-a0c5-fd12a6b2942f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:09.186509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc57f2-91b4-480b-92df-ead4908ad2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:09.187171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc57f2-91b4-480b-92df-ead4908ad2c0 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:09.187460 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc57f2-91b4-480b-92df-ead4908ad2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:09.187746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc57f2-91b4-480b-92df-ead4908ad2c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:09.205830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:09.265747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc57f2-91b4-480b-92df-ead4908ad2c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:09.267390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc57f2-91b4-480b-92df-ead4908ad2c0 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:09.340696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:09.358311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085476f-88b3-42ce-8b0d-571c1f9d0402 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:09.375320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 145/290] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:09.381891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc57f2-91b4-480b-92df-ead4908ad2c0 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:09.490726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc57f2-91b4-480b-92df-ead4908ad2c0 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:09.490893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e115827-a577-4f3e-9cc9-e52e4cb3a709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:09.491703 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115827-a577-4f3e-9cc9-e52e4cb3a709 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:09.491991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115827-a577-4f3e-9cc9-e52e4cb3a709 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:09.492271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e115827-a577-4f3e-9cc9-e52e4cb3a709 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:09.501231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e115827-a577-4f3e-9cc9-e52e4cb3a709 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:09.502777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e115827-a577-4f3e-9cc9-e52e4cb3a709 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:09.504994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 146/291] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:09 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:09.513345 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e115827-a577-4f3e-9cc9-e52e4cb3a709 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:09.529563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af2a74-fac4-44b8-bcd0-75112aa5c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:09.530463 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af2a74-fac4-44b8-bcd0-75112aa5c106 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:09.530751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af2a74-fac4-44b8-bcd0-75112aa5c106 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:09.531032 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af2a74-fac4-44b8-bcd0-75112aa5c106 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:09.532352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af2a74-fac4-44b8-bcd0-75112aa5c106 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:09.534648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af2a74-fac4-44b8-bcd0-75112aa5c106 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:09.536903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 146/292] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:09 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:09.543507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af2a74-fac4-44b8-bcd0-75112aa5c106 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:09.600735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:09.669165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af2a74-fac4-44b8-bcd0-75112aa5c106 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:09.707135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a35f5-b66c-43c8-a258-591bb4529575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:09.707831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a35f5-b66c-43c8-a258-591bb4529575 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:09.708136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a35f5-b66c-43c8-a258-591bb4529575 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:09.708489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a35f5-b66c-43c8-a258-591bb4529575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:09.709911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a35f5-b66c-43c8-a258-591bb4529575 None admin] RBAC: Adding query filter params (role.id=0f8e73b285204863a74f8af4ff7bbc86, scope.project.id=4060784cbaea460b93f362ae4cf23508, user.id=f065445e580f4df996944746ea5b32f8) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:09.710227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a35f5-b66c-43c8-a258-591bb4529575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:09.711750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a35f5-b66c-43c8-a258-591bb4529575 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:09.718597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 147/293] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:09 2022] GET /identity/v3/role_assignments?user.id=f065445e580f4df996944746ea5b32f8&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:09.763803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e115827-a577-4f3e-9cc9-e52e4cb3a709 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:09.800715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:09.812100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0267ab-de96-47bf-927c-ed571f3f0e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:09.813765 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0267ab-de96-47bf-927c-ed571f3f0e8f None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:09.814155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0267ab-de96-47bf-927c-ed571f3f0e8f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:09.814503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0267ab-de96-47bf-927c-ed571f3f0e8f None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508/users/1a2c11e17dab480d9b4cd44f25df49ef/roles/2cfac7d7d3f040f2a24b48e4f01d3eb2` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:09.816985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0267ab-de96-47bf-927c-ed571f3f0e8f None admin] RBAC: Authorizing `identity:create_grant(project_id=4060784cbaea460b93f362ae4cf23508, user_id=1a2c11e17dab480d9b4cd44f25df49ef, role_id=2cfac7d7d3f040f2a24b48e4f01d3eb2)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:09.819070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0267ab-de96-47bf-927c-ed571f3f0e8f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:09.836965 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 147/294] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:40:09 2022] PUT /identity/v3/projects/4060784cbaea460b93f362ae4cf23508/users/1a2c11e17dab480d9b4cd44f25df49ef/roles/2cfac7d7d3f040f2a24b48e4f01d3eb2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:40:10.574931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571bd02-fe0d-4a2d-9fe0-c0af25a8fdab None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:10.575607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571bd02-fe0d-4a2d-9fe0-c0af25a8fdab None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:10.575607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571bd02-fe0d-4a2d-9fe0-c0af25a8fdab None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:10.576845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 148/295] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:10.582517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bca02-1145-46eb-94ad-e9d2f9032691 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:10.583101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bca02-1145-46eb-94ad-e9d2f9032691 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:10.583101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bca02-1145-46eb-94ad-e9d2f9032691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:10.600902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-149bca02-1145-46eb-94ad-e9d2f9032691 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:10.602103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-149bca02-1145-46eb-94ad-e9d2f9032691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:10.644907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42586c-32d0-4dcd-a71c-c5e2a096dce8 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:10.645215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42586c-32d0-4dcd-a71c-c5e2a096dce8 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:10.645467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42586c-32d0-4dcd-a71c-c5e2a096dce8 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:10.647657 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 149/296] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:10.653228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebbe0-775c-4d8a-b1ff-ae19163da42c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:10.653649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebbe0-775c-4d8a-b1ff-ae19163da42c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:10.653916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebbe0-775c-4d8a-b1ff-ae19163da42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:10.672413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-a38ebbe0-775c-4d8a-b1ff-ae19163da42c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:10.673765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a38ebbe0-775c-4d8a-b1ff-ae19163da42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:10.680452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 148/297] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:10.700010 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bca02-1145-46eb-94ad-e9d2f9032691 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:10.752500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 150/298] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:10.775237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ebbe0-775c-4d8a-b1ff-ae19163da42c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:10.805067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bca02-1145-46eb-94ad-e9d2f9032691 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:10.814809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:10.832293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:10.853724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1abfa-a191-4f72-809e-19f2d52c8c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:10.854534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1abfa-a191-4f72-809e-19f2d52c8c7f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:10.854813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1abfa-a191-4f72-809e-19f2d52c8c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:10.855137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1abfa-a191-4f72-809e-19f2d52c8c7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:10.866839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1abfa-a191-4f72-809e-19f2d52c8c7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-ebd1abfa-a191-4f72-809e-19f2d52c8c7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:10.874670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:10.875787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:10.876859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 149/299] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:10.882090 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1abfa-a191-4f72-809e-19f2d52c8c7f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:10.897852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ebbe0-775c-4d8a-b1ff-ae19163da42c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:10.935939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cab60-9346-4f50-a2bf-3952165e4bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:10.936794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cab60-9346-4f50-a2bf-3952165e4bf7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:10.937081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cab60-9346-4f50-a2bf-3952165e4bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:10.937600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cab60-9346-4f50-a2bf-3952165e4bf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:10.949073 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7cab60-9346-4f50-a2bf-3952165e4bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-ec7cab60-9346-4f50-a2bf-3952165e4bf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:10.957522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:10.959037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:10.959869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 151/300] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:10.960910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41fbb-2d5b-4f4a-847b-54dc1d01ae7d None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:10.961360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41fbb-2d5b-4f4a-847b-54dc1d01ae7d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:10.961573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e41fbb-2d5b-4f4a-847b-54dc1d01ae7d None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:10.962742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 152/301] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:10.965621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e41fbb-2d5b-4f4a-847b-54dc1d01ae7d None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:11.010535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1abfa-a191-4f72-809e-19f2d52c8c7f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:11.046176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b9b97-028d-4972-a37d-67732c79464b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:11.046892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b9b97-028d-4972-a37d-67732c79464b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:11.047194 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b9b97-028d-4972-a37d-67732c79464b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:11.047485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b9b97-028d-4972-a37d-67732c79464b None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:11.048284 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b9b97-028d-4972-a37d-67732c79464b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:11.048651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b9b97-028d-4972-a37d-67732c79464b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:11.050111 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b9b97-028d-4972-a37d-67732c79464b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:11.059141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 150/302] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:11.060468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e460ff-1c37-44ed-a164-6575d18b5fb7 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:11.060761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e460ff-1c37-44ed-a164-6575d18b5fb7 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:11.061425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e460ff-1c37-44ed-a164-6575d18b5fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:11.083232 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-f0e460ff-1c37-44ed-a164-6575d18b5fb7 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:11.085477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f0e460ff-1c37-44ed-a164-6575d18b5fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:11.087590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:11.096084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e41fbb-2d5b-4f4a-847b-54dc1d01ae7d None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:11.133316 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:11.142330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bfc81-0d59-485f-bb62-4f20afa3e6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:11.143043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bfc81-0d59-485f-bb62-4f20afa3e6d7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:11.143334 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bfc81-0d59-485f-bb62-4f20afa3e6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:11.143614 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bfc81-0d59-485f-bb62-4f20afa3e6d7 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:11.144396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bfc81-0d59-485f-bb62-4f20afa3e6d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:11.144706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bfc81-0d59-485f-bb62-4f20afa3e6d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:11.146644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bfc81-0d59-485f-bb62-4f20afa3e6d7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:11.160942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 153/303] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:11.160942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bfc81-0d59-485f-bb62-4f20afa3e6d7 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:11.174821 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 151/304] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:11.175605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e460ff-1c37-44ed-a164-6575d18b5fb7 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:11.299073 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bfc81-0d59-485f-bb62-4f20afa3e6d7 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:11.315464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e460ff-1c37-44ed-a164-6575d18b5fb7 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:11.339578 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220e5b0-6ef6-4392-beb3-369abb8c94e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:11.340842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220e5b0-6ef6-4392-beb3-369abb8c94e5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:11.341405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220e5b0-6ef6-4392-beb3-369abb8c94e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:11.341689 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220e5b0-6ef6-4392-beb3-369abb8c94e5 None admin] PATH_INFO: `/v3/users/3ac419067e664910916d16535d2216d2` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:11.356868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220e5b0-6ef6-4392-beb3-369abb8c94e5 None admin] RBAC: Authorizing `identity:get_user(user_id=3ac419067e664910916d16535d2216d2)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:11.358320 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec384e-8a4a-4ff9-b42b-b01b84a94060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:11.358480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220e5b0-6ef6-4392-beb3-369abb8c94e5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:11.359245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec384e-8a4a-4ff9-b42b-b01b84a94060 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:11.359529 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec384e-8a4a-4ff9-b42b-b01b84a94060 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:11.359805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec384e-8a4a-4ff9-b42b-b01b84a94060 None admin] PATH_INFO: `/v3/users/223ab3353b9e420f882f33a055acccb6` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:11.360023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 154/305] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:11 2022] GET /identity/v3/users/3ac419067e664910916d16535d2216d2 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:11.360813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220e5b0-6ef6-4392-beb3-369abb8c94e5 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:11.361026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec384e-8a4a-4ff9-b42b-b01b84a94060 None admin] RBAC: Authorizing `identity:get_user(user_id=223ab3353b9e420f882f33a055acccb6)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:11.362447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec384e-8a4a-4ff9-b42b-b01b84a94060 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:11.368888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 152/306] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:11 2022] GET /identity/v3/users/223ab3353b9e420f882f33a055acccb6 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:11.368888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec384e-8a4a-4ff9-b42b-b01b84a94060 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:11.386944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:11.419485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:11.483059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec384e-8a4a-4ff9-b42b-b01b84a94060 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:11.483390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220e5b0-6ef6-4392-beb3-369abb8c94e5 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:11.519028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136a36-9584-462e-a92b-b51a2d07391b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:11.519687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136a36-9584-462e-a92b-b51a2d07391b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:11.519972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136a36-9584-462e-a92b-b51a2d07391b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:11.520256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136a36-9584-462e-a92b-b51a2d07391b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:11.521165 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48136a36-9584-462e-a92b-b51a2d07391b None admin] RBAC: Adding query filter params (role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2, user.id=3ac419067e664910916d16535d2216d2, scope.system=all) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:11.521480 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48136a36-9584-462e-a92b-b51a2d07391b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:11.521597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf0972-dfda-4dff-8bb2-07832282680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:11.522371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0972-dfda-4dff-8bb2-07832282680f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:11.522649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0972-dfda-4dff-8bb2-07832282680f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:11.522923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0972-dfda-4dff-8bb2-07832282680f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:11.523014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48136a36-9584-462e-a92b-b51a2d07391b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:11.524112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf0972-dfda-4dff-8bb2-07832282680f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:11.525574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf0972-dfda-4dff-8bb2-07832282680f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:11.532264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 155/307] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:11.532264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf0972-dfda-4dff-8bb2-07832282680f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:11.540726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 153/308] 10.222.0.85 () {56 vars in 1308 bytes} [Tue Jul 19 20:40:11 2022] GET /identity/v3/role_assignments?user.id=3ac419067e664910916d16535d2216d2&scope.system=all&role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:11.541626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136a36-9584-462e-a92b-b51a2d07391b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:11.657535 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:11.681622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf0972-dfda-4dff-8bb2-07832282680f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:11.697964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136a36-9584-462e-a92b-b51a2d07391b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:11.700247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:11.731437 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7439ee-9534-4fa7-9fdd-701088bca0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:11.732650 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7439ee-9534-4fa7-9fdd-701088bca0d5 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:11.733039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7439ee-9534-4fa7-9fdd-701088bca0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:11.733540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7439ee-9534-4fa7-9fdd-701088bca0d5 None admin] PATH_INFO: `/v3/system/users/223ab3353b9e420f882f33a055acccb6/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:11.735465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7439ee-9534-4fa7-9fdd-701088bca0d5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=223ab3353b9e420f882f33a055acccb6, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:11.737068 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7439ee-9534-4fa7-9fdd-701088bca0d5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:11.750715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3e020-03db-44bc-aef4-dc904a7696ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:11.751741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e020-03db-44bc-aef4-dc904a7696ca None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:11.752279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e020-03db-44bc-aef4-dc904a7696ca None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:11.752746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 156/309] 10.222.0.85 () {58 vars in 1334 bytes} [Tue Jul 19 20:40:11 2022] PUT /identity/v3/system/users/223ab3353b9e420f882f33a055acccb6/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:40:11.753166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e020-03db-44bc-aef4-dc904a7696ca None admin] PATH_INFO: `/v3/groups` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:11.761189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3e020-03db-44bc-aef4-dc904a7696ca None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:11.764557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3e020-03db-44bc-aef4-dc904a7696ca None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:11.782074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 154/310] 10.222.0.85 () {60 vars in 1134 bytes} [Tue Jul 19 20:40:11 2022] POST /identity/v3/groups => generated 229 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:40:12.079585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345c81b-9706-47ff-b861-419671b96b41 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:12.079974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345c81b-9706-47ff-b861-419671b96b41 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:12.080183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345c81b-9706-47ff-b861-419671b96b41 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:12.081822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 157/311] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:12.087810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4b48-96ae-4ad2-bd4a-48454bdddd57 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:12.088098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4b48-96ae-4ad2-bd4a-48454bdddd57 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:12.088403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d4b48-96ae-4ad2-bd4a-48454bdddd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:12.105801 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-128d4b48-96ae-4ad2-bd4a-48454bdddd57 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:12.107070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-128d4b48-96ae-4ad2-bd4a-48454bdddd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:12.179936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 155/312] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:12.205526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345c81b-9706-47ff-b861-419671b96b41 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:12.336477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345c81b-9706-47ff-b861-419671b96b41 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:12.372643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4395d-8d87-43e3-b432-26fd4f3ec7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:12.373459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4395d-8d87-43e3-b432-26fd4f3ec7eb None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:12.373746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4395d-8d87-43e3-b432-26fd4f3ec7eb None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:12.374082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4395d-8d87-43e3-b432-26fd4f3ec7eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:12.385900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4395d-8d87-43e3-b432-26fd4f3ec7eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-d8c4395d-8d87-43e3-b432-26fd4f3ec7eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:12.393833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:12.394748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:12.395613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 158/313] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:12.401035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d4b48-96ae-4ad2-bd4a-48454bdddd57 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:12.503905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:12.515069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d4b48-96ae-4ad2-bd4a-48454bdddd57 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:12.554309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dcef7-d862-4714-a4f2-b21fb6b9dccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:12.555030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dcef7-d862-4714-a4f2-b21fb6b9dccf None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:12.555319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dcef7-d862-4714-a4f2-b21fb6b9dccf None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:12.555619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dcef7-d862-4714-a4f2-b21fb6b9dccf None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:12.556697 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dcef7-d862-4714-a4f2-b21fb6b9dccf None admin] RBAC: Adding query filter params (name=reader) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:12.557030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dcef7-d862-4714-a4f2-b21fb6b9dccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:12.558628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dcef7-d862-4714-a4f2-b21fb6b9dccf None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:12.568060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 156/314] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:12.574070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4395d-8d87-43e3-b432-26fd4f3ec7eb None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:12.584906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:12.686153 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4395d-8d87-43e3-b432-26fd4f3ec7eb None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:12.723748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a2d12-3a82-4a3c-a1ec-ae25b215c96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:12.725527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a2d12-3a82-4a3c-a1ec-ae25b215c96b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:12.726011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a2d12-3a82-4a3c-a1ec-ae25b215c96b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:12.726506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a2d12-3a82-4a3c-a1ec-ae25b215c96b None admin] PATH_INFO: `/v3/users/1a2c11e17dab480d9b4cd44f25df49ef` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:12.727964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a2d12-3a82-4a3c-a1ec-ae25b215c96b None admin] RBAC: Authorizing `identity:get_user(user_id=1a2c11e17dab480d9b4cd44f25df49ef)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:12.729647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a2d12-3a82-4a3c-a1ec-ae25b215c96b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:12.731853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 159/315] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:12 2022] GET /identity/v3/users/1a2c11e17dab480d9b4cd44f25df49ef => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:12.740396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dcef7-d862-4714-a4f2-b21fb6b9dccf None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:12.851191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dcef7-d862-4714-a4f2-b21fb6b9dccf None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:12.887001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67d6a0-6bbe-4c7a-a7f7-a91bcb8306f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:12.887887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67d6a0-6bbe-4c7a-a7f7-a91bcb8306f8 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:12.888363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67d6a0-6bbe-4c7a-a7f7-a91bcb8306f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:12.888788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67d6a0-6bbe-4c7a-a7f7-a91bcb8306f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:12.915448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:12.968760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67d6a0-6bbe-4c7a-a7f7-a91bcb8306f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:12.970464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67d6a0-6bbe-4c7a-a7f7-a91bcb8306f8 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:13.084234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 157/316] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:13.092391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a2d12-3a82-4a3c-a1ec-ae25b215c96b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:13.181858 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:13.228842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a2d12-3a82-4a3c-a1ec-ae25b215c96b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:13.276192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0edb0-f0b9-495e-99ae-d951c76ac101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:13.277118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0edb0-f0b9-495e-99ae-d951c76ac101 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:13.277404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0edb0-f0b9-495e-99ae-d951c76ac101 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:13.277685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0edb0-f0b9-495e-99ae-d951c76ac101 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:13.278955 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0edb0-f0b9-495e-99ae-d951c76ac101 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:13.280419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0edb0-f0b9-495e-99ae-d951c76ac101 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:13.282130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 160/317] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:13 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:13.288106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67d6a0-6bbe-4c7a-a7f7-a91bcb8306f8 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:13.352734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:13.402678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67d6a0-6bbe-4c7a-a7f7-a91bcb8306f8 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:13.439901 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4265f1-80cb-4826-9bf8-f866b339a481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:13.440889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4265f1-80cb-4826-9bf8-f866b339a481 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:13.441347 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4265f1-80cb-4826-9bf8-f866b339a481 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:13.441828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4265f1-80cb-4826-9bf8-f866b339a481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:13.443351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4265f1-80cb-4826-9bf8-f866b339a481 None admin] RBAC: Adding query filter params (role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2, scope.project.id=4060784cbaea460b93f362ae4cf23508, user.id=1a2c11e17dab480d9b4cd44f25df49ef) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:13.443889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4265f1-80cb-4826-9bf8-f866b339a481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:13.445523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4265f1-80cb-4826-9bf8-f866b339a481 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:13.452552 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 158/318] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:13 2022] GET /identity/v3/role_assignments?user.id=1a2c11e17dab480d9b4cd44f25df49ef&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:13.822925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22356cfc-c070-4fe9-bb2a-1ff3c19f42fc None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:13.823491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22356cfc-c070-4fe9-bb2a-1ff3c19f42fc None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:13.823491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22356cfc-c070-4fe9-bb2a-1ff3c19f42fc None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:13.825395 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 161/319] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:13.830925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123d413-b44b-4d2b-bc57-2b035613b290 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:13.831392 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123d413-b44b-4d2b-bc57-2b035613b290 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:13.831493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123d413-b44b-4d2b-bc57-2b035613b290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:13.852678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-0123d413-b44b-4d2b-bc57-2b035613b290 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:13.853950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0123d413-b44b-4d2b-bc57-2b035613b290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:13.926264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 159/320] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:13.948279 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22356cfc-c070-4fe9-bb2a-1ff3c19f42fc None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:13.972249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ed380-e991-42a4-bb0f-83543fb414f8 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:13.973125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ed380-e991-42a4-bb0f-83543fb414f8 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:13.973125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ed380-e991-42a4-bb0f-83543fb414f8 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:13.975212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 160/321] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:40:13.980242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f16a9-961d-4e9f-aea5-d8b785281046 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:13.980554 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f16a9-961d-4e9f-aea5-d8b785281046 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:13.980835 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f16a9-961d-4e9f-aea5-d8b785281046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:13.998083 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-b57f16a9-961d-4e9f-aea5-d8b785281046 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:13.999274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b57f16a9-961d-4e9f-aea5-d8b785281046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:14.048088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:14.063108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22356cfc-c070-4fe9-bb2a-1ff3c19f42fc None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:14.075829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 161/322] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:40:14.075829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a89c-c10b-41d5-8f84-894ed5fbd8e1 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:14.075829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a89c-c10b-41d5-8f84-894ed5fbd8e1 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:14.075829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a89c-c10b-41d5-8f84-894ed5fbd8e1 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:14.077119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 162/323] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:14.082421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2aecd-38f5-4b60-9f47-001975f1a061 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:14.082682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2aecd-38f5-4b60-9f47-001975f1a061 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:14.082957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2aecd-38f5-4b60-9f47-001975f1a061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:14.100006 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c9a2aecd-38f5-4b60-9f47-001975f1a061 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:14.101342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c9a2aecd-38f5-4b60-9f47-001975f1a061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:14.101575 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fef72-710e-4774-8422-af6e94a8cd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:14.102382 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fef72-710e-4774-8422-af6e94a8cd7a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:14.102665 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fef72-710e-4774-8422-af6e94a8cd7a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:14.102944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fef72-710e-4774-8422-af6e94a8cd7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:14.113974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fef72-710e-4774-8422-af6e94a8cd7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-da3fef72-710e-4774-8422-af6e94a8cd7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:14.121820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:14.122748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:14.123563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 162/324] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:14.124233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fef72-710e-4774-8422-af6e94a8cd7a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:14.126678 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:14.175358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 163/325] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:40:14.176202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2aecd-38f5-4b60-9f47-001975f1a061 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:14.236672 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fef72-710e-4774-8422-af6e94a8cd7a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:14.272615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b26cc-bf6d-4a88-b73d-da051da5bbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:14.273450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b26cc-bf6d-4a88-b73d-da051da5bbde None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:14.273868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b26cc-bf6d-4a88-b73d-da051da5bbde None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:14.274147 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b26cc-bf6d-4a88-b73d-da051da5bbde None admin] PATH_INFO: `/v3/domains/default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:14.275357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b26cc-bf6d-4a88-b73d-da051da5bbde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:14.277060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b26cc-bf6d-4a88-b73d-da051da5bbde None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:14.278776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 163/326] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:14.279553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b26cc-bf6d-4a88-b73d-da051da5bbde None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:14.284950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2aecd-38f5-4b60-9f47-001975f1a061 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:14.295084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:14.321676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145c34f-06c1-415c-9992-ef4057e78c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:14.322378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145c34f-06c1-415c-9992-ef4057e78c95 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:14.322665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145c34f-06c1-415c-9992-ef4057e78c95 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:14.322937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145c34f-06c1-415c-9992-ef4057e78c95 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:14.323682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145c34f-06c1-415c-9992-ef4057e78c95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:14.323990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145c34f-06c1-415c-9992-ef4057e78c95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:14.325513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145c34f-06c1-415c-9992-ef4057e78c95 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:14.334719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 164/327] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:14.335287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145c34f-06c1-415c-9992-ef4057e78c95 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:14.361978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:14.393517 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b26cc-bf6d-4a88-b73d-da051da5bbde None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:14.428836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ffa14-a5b1-4b63-bee3-66cfac1adf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:14.429628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ffa14-a5b1-4b63-bee3-66cfac1adf98 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:14.429916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ffa14-a5b1-4b63-bee3-66cfac1adf98 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:14.430190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ffa14-a5b1-4b63-bee3-66cfac1adf98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:14.440495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ffa14-a5b1-4b63-bee3-66cfac1adf98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:14.441906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145c34f-06c1-415c-9992-ef4057e78c95 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-bd7ffa14-a5b1-4b63-bee3-66cfac1adf98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:14.448705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:14.449595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:14.450405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 164/328] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:14.451065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ffa14-a5b1-4b63-bee3-66cfac1adf98 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:14.478229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05dc23-4cc5-42b7-8294-1482831d8f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:14.478902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05dc23-4cc5-42b7-8294-1482831d8f89 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:14.479189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05dc23-4cc5-42b7-8294-1482831d8f89 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:14.479468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05dc23-4cc5-42b7-8294-1482831d8f89 None admin] PATH_INFO: `/v3/groups` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:14.480154 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05dc23-4cc5-42b7-8294-1482831d8f89 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:14.481648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05dc23-4cc5-42b7-8294-1482831d8f89 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:14.495810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 165/329] 10.222.0.85 () {60 vars in 1134 bytes} [Tue Jul 19 20:40:14 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:40:14.496660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05dc23-4cc5-42b7-8294-1482831d8f89 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:14.532970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:14.580177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ffa14-a5b1-4b63-bee3-66cfac1adf98 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:14.614003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:14.619513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05dc23-4cc5-42b7-8294-1482831d8f89 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:14.622996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428f612-51e7-4a68-a218-e06f0936d9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:14.623636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428f612-51e7-4a68-a218-e06f0936d9b8 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:14.623921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428f612-51e7-4a68-a218-e06f0936d9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:14.624197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428f612-51e7-4a68-a218-e06f0936d9b8 None admin] PATH_INFO: `/v3/users/3ac419067e664910916d16535d2216d2` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:14.625525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428f612-51e7-4a68-a218-e06f0936d9b8 None admin] RBAC: Authorizing `identity:get_user(user_id=3ac419067e664910916d16535d2216d2)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:14.626996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428f612-51e7-4a68-a218-e06f0936d9b8 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:14.628925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 165/330] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:14 2022] GET /identity/v3/users/3ac419067e664910916d16535d2216d2 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:14.635551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428f612-51e7-4a68-a218-e06f0936d9b8 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:14.659383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835de3c-6843-4754-bbc7-44905b512428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:14.660373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835de3c-6843-4754-bbc7-44905b512428 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:14.660520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835de3c-6843-4754-bbc7-44905b512428 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:14.661045 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835de3c-6843-4754-bbc7-44905b512428 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:14.661414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835de3c-6843-4754-bbc7-44905b512428 None admin] RBAC: Adding query filter params (name=member) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:14.661752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835de3c-6843-4754-bbc7-44905b512428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:14.663143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835de3c-6843-4754-bbc7-44905b512428 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:14.674218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 166/331] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:14.679963 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835de3c-6843-4754-bbc7-44905b512428 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:14.770866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428f612-51e7-4a68-a218-e06f0936d9b8 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:14.804548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:14.816625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835de3c-6843-4754-bbc7-44905b512428 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:14.825408 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f42fb-2cba-4a25-afb5-bb594efa5535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:14.826915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f42fb-2cba-4a25-afb5-bb594efa5535 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:14.827599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f42fb-2cba-4a25-afb5-bb594efa5535 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:14.827891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f42fb-2cba-4a25-afb5-bb594efa5535 None admin] PATH_INFO: `/v3/system/users/3ac419067e664910916d16535d2216d2/roles/2cfac7d7d3f040f2a24b48e4f01d3eb2` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:14.830856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f42fb-2cba-4a25-afb5-bb594efa5535 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3ac419067e664910916d16535d2216d2, role_id=2cfac7d7d3f040f2a24b48e4f01d3eb2)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:14.832487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f42fb-2cba-4a25-afb5-bb594efa5535 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:14.841865 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 166/332] 10.222.0.85 () {58 vars in 1334 bytes} [Tue Jul 19 20:40:14 2022] PUT /identity/v3/system/users/3ac419067e664910916d16535d2216d2/roles/2cfac7d7d3f040f2a24b48e4f01d3eb2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:40:14.864199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97106a0a-8a92-4888-98ca-2a677975a7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:14.865684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97106a0a-8a92-4888-98ca-2a677975a7c7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:14.866028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97106a0a-8a92-4888-98ca-2a677975a7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:14.866337 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97106a0a-8a92-4888-98ca-2a677975a7c7 None admin] PATH_INFO: `/v3/users/223ab3353b9e420f882f33a055acccb6` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:14.867782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97106a0a-8a92-4888-98ca-2a677975a7c7 None admin] RBAC: Authorizing `identity:get_user(user_id=223ab3353b9e420f882f33a055acccb6)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:14.869415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97106a0a-8a92-4888-98ca-2a677975a7c7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:14.871693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 167/333] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:14 2022] GET /identity/v3/users/223ab3353b9e420f882f33a055acccb6 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:14.877852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f42fb-2cba-4a25-afb5-bb594efa5535 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:14.959923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:15.009190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f42fb-2cba-4a25-afb5-bb594efa5535 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:15.047884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32835cab-e5c0-4c78-9422-6288c422849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:15.048588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835cab-e5c0-4c78-9422-6288c422849e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:15.048878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835cab-e5c0-4c78-9422-6288c422849e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:15.049162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32835cab-e5c0-4c78-9422-6288c422849e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:15.050010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32835cab-e5c0-4c78-9422-6288c422849e None admin] RBAC: Adding query filter params (role.id=0f8e73b285204863a74f8af4ff7bbc86, user.id=223ab3353b9e420f882f33a055acccb6, scope.system=all) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:15.050327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32835cab-e5c0-4c78-9422-6288c422849e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:15.051783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32835cab-e5c0-4c78-9422-6288c422849e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:15.066456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 167/334] 10.222.0.85 () {56 vars in 1308 bytes} [Tue Jul 19 20:40:14 2022] GET /identity/v3/role_assignments?user.id=223ab3353b9e420f882f33a055acccb6&scope.system=all&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:17.236255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9da5-e9b5-44d2-9918-f1a1d137058c None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:17.236885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9da5-e9b5-44d2-9918-f1a1d137058c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:17.236885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9da5-e9b5-44d2-9918-f1a1d137058c None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:17.238240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 168/335] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:17.244215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b31d-0b93-41d9-8e63-28e61bb51122 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:17.244731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b31d-0b93-41d9-8e63-28e61bb51122 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:17.244812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b31d-0b93-41d9-8e63-28e61bb51122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:17.265404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-5094b31d-0b93-41d9-8e63-28e61bb51122 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:17.266635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-5094b31d-0b93-41d9-8e63-28e61bb51122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:17.350685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 168/336] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:17.393417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa9da5-e9b5-44d2-9918-f1a1d137058c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:17.439841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9bb76-8722-4340-a9a8-72e1104586ba None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:17.440175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9bb76-8722-4340-a9a8-72e1104586ba None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:17.440432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9bb76-8722-4340-a9a8-72e1104586ba None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:17.441995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 169/337] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:40:17.448196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf2777-1801-45b8-829d-9e2791dc6f4b None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:17.448904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf2777-1801-45b8-829d-9e2791dc6f4b None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:17.449401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf2777-1801-45b8-829d-9e2791dc6f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:17.474754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-08cf2777-1801-45b8-829d-9e2791dc6f4b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:17.478212 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-08cf2777-1801-45b8-829d-9e2791dc6f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:17.547327 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa9da5-e9b5-44d2-9918-f1a1d137058c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:17.567579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:17.601474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 170/338] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:17.602818 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682925c-813a-4171-8550-f51c92d1e9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:17.603998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682925c-813a-4171-8550-f51c92d1e9bf None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:17.604533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682925c-813a-4171-8550-f51c92d1e9bf None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:17.605036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682925c-813a-4171-8550-f51c92d1e9bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:17.619771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682925c-813a-4171-8550-f51c92d1e9bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-2682925c-813a-4171-8550-f51c92d1e9bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:17.630319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:17.631815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:17.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:17.635662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 169/339] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:40:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:17.644188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf2777-1801-45b8-829d-9e2791dc6f4b None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:17.658015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682925c-813a-4171-8550-f51c92d1e9bf None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:17.696358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:17.797646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf2777-1801-45b8-829d-9e2791dc6f4b None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:17.823878 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682925c-813a-4171-8550-f51c92d1e9bf None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:17.840681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2fa2c-57e6-4dd6-895d-fa6b9a2315d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:17.841978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2fa2c-57e6-4dd6-895d-fa6b9a2315d1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:17.841978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2fa2c-57e6-4dd6-895d-fa6b9a2315d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:17.841978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2fa2c-57e6-4dd6-895d-fa6b9a2315d1 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:17.842481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2fa2c-57e6-4dd6-895d-fa6b9a2315d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:17.842852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2fa2c-57e6-4dd6-895d-fa6b9a2315d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:17.845335 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2fa2c-57e6-4dd6-895d-fa6b9a2315d1 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:17.855128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 171/340] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:17.856670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183180b-d69d-427e-ad8c-7f70092fe74f None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:17.856809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183180b-d69d-427e-ad8c-7f70092fe74f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:17.857058 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183180b-d69d-427e-ad8c-7f70092fe74f None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:17.858378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 172/341] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:17.859527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17adf6-8dad-4ea6-82cc-0160526de2d4 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:17.859915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17adf6-8dad-4ea6-82cc-0160526de2d4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:17.860082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17adf6-8dad-4ea6-82cc-0160526de2d4 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:17.861386 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 173/342] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:17.865952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae6a1e-d694-425c-89af-87688e4cfa10 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:17.866245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae6a1e-d694-425c-89af-87688e4cfa10 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:17.866529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae6a1e-d694-425c-89af-87688e4cfa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:17.871257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea18e83-3710-40ed-a5b3-ba8d7734c2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:17.872443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea18e83-3710-40ed-a5b3-ba8d7734c2f6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:17.872921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea18e83-3710-40ed-a5b3-ba8d7734c2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:17.873479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea18e83-3710-40ed-a5b3-ba8d7734c2f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:17.889774 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-fdae6a1e-d694-425c-89af-87688e4cfa10 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:17.891156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fdae6a1e-d694-425c-89af-87688e4cfa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:17.893534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea18e83-3710-40ed-a5b3-ba8d7734c2f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-6ea18e83-3710-40ed-a5b3-ba8d7734c2f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:17.906400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:17.907706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:17.908872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 170/343] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:17.909790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea18e83-3710-40ed-a5b3-ba8d7734c2f6 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:17.940830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:18.005870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 174/344] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:18.010634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6968e-ff2e-46d9-9af9-9e1044b4ad05 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:18.010634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6968e-ff2e-46d9-9af9-9e1044b4ad05 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:18.010634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6968e-ff2e-46d9-9af9-9e1044b4ad05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:18.037467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-42f6968e-ff2e-46d9-9af9-9e1044b4ad05 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:18.038812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-42f6968e-ff2e-46d9-9af9-9e1044b4ad05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:18.066331 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:18.092536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea18e83-3710-40ed-a5b3-ba8d7734c2f6 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:18.127951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 175/345] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:18.128791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6968e-ff2e-46d9-9af9-9e1044b4ad05 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:18.143758 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53d7ba-fda7-4f51-9c62-134613b94abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:18.144908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53d7ba-fda7-4f51-9c62-134613b94abd None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:18.145468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53d7ba-fda7-4f51-9c62-134613b94abd None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:18.146486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53d7ba-fda7-4f51-9c62-134613b94abd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:18.231629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53d7ba-fda7-4f51-9c62-134613b94abd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:18.233491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53d7ba-fda7-4f51-9c62-134613b94abd None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:18.264368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6968e-ff2e-46d9-9af9-9e1044b4ad05 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:18.266000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:18.315543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d7a91-d626-4f66-8dfc-aaae372ec76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:18.316700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d7a91-d626-4f66-8dfc-aaae372ec76e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:18.317203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d7a91-d626-4f66-8dfc-aaae372ec76e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:18.317776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d7a91-d626-4f66-8dfc-aaae372ec76e None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:18.319309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d7a91-d626-4f66-8dfc-aaae372ec76e None admin] RBAC: Adding query filter params (name=member) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:18.319827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d7a91-d626-4f66-8dfc-aaae372ec76e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:18.321438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d7a91-d626-4f66-8dfc-aaae372ec76e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:18.332509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 176/346] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:18.333710 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d7a91-d626-4f66-8dfc-aaae372ec76e None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:18.369307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:18.397026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 171/347] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:18.397899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53d7ba-fda7-4f51-9c62-134613b94abd None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:18.457762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d7a91-d626-4f66-8dfc-aaae372ec76e None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:18.509712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a104298-654b-4af0-b999-088c6a13f221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:18.510919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a104298-654b-4af0-b999-088c6a13f221 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:18.511375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a104298-654b-4af0-b999-088c6a13f221 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:18.511818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a104298-654b-4af0-b999-088c6a13f221 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:18.518708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53d7ba-fda7-4f51-9c62-134613b94abd None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:18.525925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a104298-654b-4af0-b999-088c6a13f221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-2a104298-654b-4af0-b999-088c6a13f221 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:18.535598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:18.537317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:18.538237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:18.540252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 177/348] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:18.541358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a104298-654b-4af0-b999-088c6a13f221 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:18.558426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b98f2a-dc9b-4a9f-8b01-8d046c6aa8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:18.559292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b98f2a-dc9b-4a9f-8b01-8d046c6aa8ec None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:18.559576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b98f2a-dc9b-4a9f-8b01-8d046c6aa8ec None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:18.559852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b98f2a-dc9b-4a9f-8b01-8d046c6aa8ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:18.572229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b98f2a-dc9b-4a9f-8b01-8d046c6aa8ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-c2b98f2a-dc9b-4a9f-8b01-8d046c6aa8ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:18.581162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:18.582307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 20:40:18.583314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:18.584721 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:18.584806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 172/349] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:18.584806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b98f2a-dc9b-4a9f-8b01-8d046c6aa8ec None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:18.633541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:18.667668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a104298-654b-4af0-b999-088c6a13f221 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:18.700508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b98f2a-dc9b-4a9f-8b01-8d046c6aa8ec None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:18.708618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c58b4e-e904-4295-9cb9-955c7d2b879d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:18.709532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58b4e-e904-4295-9cb9-955c7d2b879d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:18.709976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58b4e-e904-4295-9cb9-955c7d2b879d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:18.710412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58b4e-e904-4295-9cb9-955c7d2b879d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:18.741653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e666b-d93a-4866-ac6a-e29255cee1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:18.742759 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e666b-d93a-4866-ac6a-e29255cee1a7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:18.743036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e666b-d93a-4866-ac6a-e29255cee1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:18.743388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e666b-d93a-4866-ac6a-e29255cee1a7 None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:18.752621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e666b-d93a-4866-ac6a-e29255cee1a7 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:18.752621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e666b-d93a-4866-ac6a-e29255cee1a7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:18.752621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 173/350] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:18 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:18.752621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e666b-d93a-4866-ac6a-e29255cee1a7 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:18.804010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c58b4e-e904-4295-9cb9-955c7d2b879d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:18.805884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c58b4e-e904-4295-9cb9-955c7d2b879d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:18.833126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:18.877068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e666b-d93a-4866-ac6a-e29255cee1a7 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:18.886147 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:18.919687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b2032-491a-4c3c-b18a-ae57b44dfb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:18.920402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b2032-491a-4c3c-b18a-ae57b44dfb7d None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:18.920693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b2032-491a-4c3c-b18a-ae57b44dfb7d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:18.921286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b2032-491a-4c3c-b18a-ae57b44dfb7d None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:18.921727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b2032-491a-4c3c-b18a-ae57b44dfb7d None admin] RBAC: Adding query filter params (name=member) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:18.922041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b2032-491a-4c3c-b18a-ae57b44dfb7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:18.923420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b2032-491a-4c3c-b18a-ae57b44dfb7d None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:18.934324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 174/351] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:18.934985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b2032-491a-4c3c-b18a-ae57b44dfb7d None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:18.938506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 178/352] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:18.939367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c58b4e-e904-4295-9cb9-955c7d2b879d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:19.075518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b2032-491a-4c3c-b18a-ae57b44dfb7d None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:19.088890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c58b4e-e904-4295-9cb9-955c7d2b879d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:19.123188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:19.143108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ad0fc-406c-45a3-bf89-8d92c0c6aced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:19.143496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16646607-7f40-4c08-bcae-0bf888330150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:19.144636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ad0fc-406c-45a3-bf89-8d92c0c6aced None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.144767 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646607-7f40-4c08-bcae-0bf888330150 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.145049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ad0fc-406c-45a3-bf89-8d92c0c6aced None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.145125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646607-7f40-4c08-bcae-0bf888330150 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.145193 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ad0fc-406c-45a3-bf89-8d92c0c6aced None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.145375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646607-7f40-4c08-bcae-0bf888330150 None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.145561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ad0fc-406c-45a3-bf89-8d92c0c6aced None admin] RBAC: Adding query filter params (name=reader) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:19.146409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ad0fc-406c-45a3-bf89-8d92c0c6aced None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:19.148066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ad0fc-406c-45a3-bf89-8d92c0c6aced None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:19.155498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16646607-7f40-4c08-bcae-0bf888330150 None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:19.157210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16646607-7f40-4c08-bcae-0bf888330150 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:19.159153 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 179/353] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:18 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:19.161347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c3823-fe22-4f4b-b540-675a23d9f070 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.161347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c3823-fe22-4f4b-b540-675a23d9f070 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.161347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c3823-fe22-4f4b-b540-675a23d9f070 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.163030 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 180/354] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:19.164031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c3823-fe22-4f4b-b540-675a23d9f070 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:19.170883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 175/355] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:19.172528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ad0fc-406c-45a3-bf89-8d92c0c6aced None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:19.234471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:19.322895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c3823-fe22-4f4b-b540-675a23d9f070 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:19.330125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ad0fc-406c-45a3-bf89-8d92c0c6aced None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:19.378387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8bee5-20bd-48ab-b43e-79a6ffb7a541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:19.379799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8bee5-20bd-48ab-b43e-79a6ffb7a541 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.380885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8bee5-20bd-48ab-b43e-79a6ffb7a541 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.381311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8bee5-20bd-48ab-b43e-79a6ffb7a541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.386402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc6e1d-f578-4e67-a619-f944c9cf822b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:19.388578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc6e1d-f578-4e67-a619-f944c9cf822b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.389054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc6e1d-f578-4e67-a619-f944c9cf822b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.389414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc6e1d-f578-4e67-a619-f944c9cf822b None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.392733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc6e1d-f578-4e67-a619-f944c9cf822b None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:19.394149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc6e1d-f578-4e67-a619-f944c9cf822b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:19.400415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 176/356] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:19 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:19.400415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc6e1d-f578-4e67-a619-f944c9cf822b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:19.487444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:19.504412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8bee5-20bd-48ab-b43e-79a6ffb7a541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:19.506705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8bee5-20bd-48ab-b43e-79a6ffb7a541 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:19.564552 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc6e1d-f578-4e67-a619-f944c9cf822b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:19.566304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:19.607103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44681413-2520-44ae-b7d5-6a8f2e9ef4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:19.608098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681413-2520-44ae-b7d5-6a8f2e9ef4fb None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.608550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681413-2520-44ae-b7d5-6a8f2e9ef4fb None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.608839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681413-2520-44ae-b7d5-6a8f2e9ef4fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.609684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44681413-2520-44ae-b7d5-6a8f2e9ef4fb None admin] RBAC: Adding query filter params (group.id=bf1c6135868f438e9f7f34737cf60e53, role.id=aab847245001401cb4d3d43238298f07, scope.project.id=e1b269e3c66347e1aa9595854069b963) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:19.609998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44681413-2520-44ae-b7d5-6a8f2e9ef4fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:19.611496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44681413-2520-44ae-b7d5-6a8f2e9ef4fb None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:19.617827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 177/357] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jul 19 20:40:19 2022] GET /identity/v3/role_assignments?group.id=bf1c6135868f438e9f7f34737cf60e53&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=aab847245001401cb4d3d43238298f07 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:19.619734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47332afe-0bdd-4ffe-9bdd-eb7113e39ae5 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.620029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47332afe-0bdd-4ffe-9bdd-eb7113e39ae5 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.620311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47332afe-0bdd-4ffe-9bdd-eb7113e39ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.641134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 181/358] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:19.642427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8bee5-20bd-48ab-b43e-79a6ffb7a541 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:19.654953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-47332afe-0bdd-4ffe-9bdd-eb7113e39ae5 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:19.656226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-47332afe-0bdd-4ffe-9bdd-eb7113e39ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:19.759931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 178/359] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:19.760982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47332afe-0bdd-4ffe-9bdd-eb7113e39ae5 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:19.769066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:19.786384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8bee5-20bd-48ab-b43e-79a6ffb7a541 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:19.833804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61a130-1302-4a34-a270-0a3df9e7282c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:19.835404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61a130-1302-4a34-a270-0a3df9e7282c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.835901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61a130-1302-4a34-a270-0a3df9e7282c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.839418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61a130-1302-4a34-a270-0a3df9e7282c None admin] PATH_INFO: `/v3/users/3ac419067e664910916d16535d2216d2` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.839418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61a130-1302-4a34-a270-0a3df9e7282c None admin] RBAC: Authorizing `identity:get_user(user_id=3ac419067e664910916d16535d2216d2)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:19.841886 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61a130-1302-4a34-a270-0a3df9e7282c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:19.842096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 182/360] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:40:19 2022] GET /identity/v3/users/3ac419067e664910916d16535d2216d2 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:19.843353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510ca7e-7c29-4ebe-900f-33e1e8a2b6e0 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.843645 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510ca7e-7c29-4ebe-900f-33e1e8a2b6e0 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.843927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510ca7e-7c29-4ebe-900f-33e1e8a2b6e0 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.845414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 183/361] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:19.845998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510ca7e-7c29-4ebe-900f-33e1e8a2b6e0 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:19.871108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:19.884227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47332afe-0bdd-4ffe-9bdd-eb7113e39ae5 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:19.928007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b864aa-715b-4546-baa5-69b0b370e339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:19.928883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b864aa-715b-4546-baa5-69b0b370e339 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:19.929167 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b864aa-715b-4546-baa5-69b0b370e339 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:19.929442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b864aa-715b-4546-baa5-69b0b370e339 None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:19.930699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b864aa-715b-4546-baa5-69b0b370e339 None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:19.932671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b864aa-715b-4546-baa5-69b0b370e339 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:19.934792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 179/362] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:19 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:19.935774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b864aa-715b-4546-baa5-69b0b370e339 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:19.986087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510ca7e-7c29-4ebe-900f-33e1e8a2b6e0 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:20.036128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ab684-120c-48cc-af38-3b4cc1813ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:20.038368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ab684-120c-48cc-af38-3b4cc1813ced None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:20.038716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ab684-120c-48cc-af38-3b4cc1813ced None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:20.039004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ab684-120c-48cc-af38-3b4cc1813ced None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:20.040362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ab684-120c-48cc-af38-3b4cc1813ced None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:20.042868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ab684-120c-48cc-af38-3b4cc1813ced None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:20.048456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 184/363] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:19 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:20.049380 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ab684-120c-48cc-af38-3b4cc1813ced None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:20.085422 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b864aa-715b-4546-baa5-69b0b370e339 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:20.096185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:20.137918 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cfbba-7650-4b3b-9303-e9fb0e004916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:20.138764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cfbba-7650-4b3b-9303-e9fb0e004916 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:20.139056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cfbba-7650-4b3b-9303-e9fb0e004916 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:20.139325 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cfbba-7650-4b3b-9303-e9fb0e004916 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:20.153725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806cfbba-7650-4b3b-9303-e9fb0e004916 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-806cfbba-7650-4b3b-9303-e9fb0e004916 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:20.164904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:20.165942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:20.166932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:20.169638 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 180/364] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:40:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:20.171029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dee096-36d0-41e2-90b3-6e733fde621f None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:20.171443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dee096-36d0-41e2-90b3-6e733fde621f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:20.171864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dee096-36d0-41e2-90b3-6e733fde621f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:20.192244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-f3dee096-36d0-41e2-90b3-6e733fde621f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:20.193669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f3dee096-36d0-41e2-90b3-6e733fde621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:20.211666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ab684-120c-48cc-af38-3b4cc1813ced None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:20.221234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:20.261506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf592c4-4a32-417c-9785-d908daa7e638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:20.262170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf592c4-4a32-417c-9785-d908daa7e638 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:20.262461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf592c4-4a32-417c-9785-d908daa7e638 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:20.262733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf592c4-4a32-417c-9785-d908daa7e638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:20.263564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf592c4-4a32-417c-9785-d908daa7e638 None admin] RBAC: Adding query filter params (role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2, user.id=3ac419067e664910916d16535d2216d2, scope.system=all) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:20.263875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf592c4-4a32-417c-9785-d908daa7e638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:20.265450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf592c4-4a32-417c-9785-d908daa7e638 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:20.278249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 185/365] 10.222.0.85 () {56 vars in 1308 bytes} [Tue Jul 19 20:40:20 2022] GET /identity/v3/role_assignments?user.id=3ac419067e664910916d16535d2216d2&scope.system=all&role.id=2cfac7d7d3f040f2a24b48e4f01d3eb2 => generated 510 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:20.278928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf592c4-4a32-417c-9785-d908daa7e638 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:20.317502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 181/366] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:20.319007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dee096-36d0-41e2-90b3-6e733fde621f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:20.409792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:20.447839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf592c4-4a32-417c-9785-d908daa7e638 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:20.489086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dee096-36d0-41e2-90b3-6e733fde621f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:20.510143 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146df6f-8bae-4ca7-87f0-56130e3e071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:20.510869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146df6f-8bae-4ca7-87f0-56130e3e071b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:20.511162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146df6f-8bae-4ca7-87f0-56130e3e071b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:20.511440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146df6f-8bae-4ca7-87f0-56130e3e071b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:20.512281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146df6f-8bae-4ca7-87f0-56130e3e071b None admin] RBAC: Adding query filter params (group.id=bf1c6135868f438e9f7f34737cf60e53, role.id=0f8e73b285204863a74f8af4ff7bbc86, scope.project.id=e1b269e3c66347e1aa9595854069b963) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:20.512670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146df6f-8bae-4ca7-87f0-56130e3e071b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:20.514253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5146df6f-8bae-4ca7-87f0-56130e3e071b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:20.522141 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 186/367] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jul 19 20:40:20 2022] GET /identity/v3/role_assignments?group.id=bf1c6135868f438e9f7f34737cf60e53&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 260 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:20.523040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146df6f-8bae-4ca7-87f0-56130e3e071b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:20.551074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d3a8d-d8f9-41f7-80b5-bc8a28bf1ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:20.552197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d3a8d-d8f9-41f7-80b5-bc8a28bf1ee0 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:20.553805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d3a8d-d8f9-41f7-80b5-bc8a28bf1ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:20.554217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d3a8d-d8f9-41f7-80b5-bc8a28bf1ee0 None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:20.555772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d3a8d-d8f9-41f7-80b5-bc8a28bf1ee0 None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:20.557562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d3a8d-d8f9-41f7-80b5-bc8a28bf1ee0 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:20.559512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 182/368] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:20 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:20.560299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d3a8d-d8f9-41f7-80b5-bc8a28bf1ee0 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:20.602618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:20.670317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146df6f-8bae-4ca7-87f0-56130e3e071b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:20.731364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d3a8d-d8f9-41f7-80b5-bc8a28bf1ee0 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:20.734029 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f839d8-24b5-461b-81cd-f6322a90c442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:20.734733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f839d8-24b5-461b-81cd-f6322a90c442 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:20.735015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f839d8-24b5-461b-81cd-f6322a90c442 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:20.735290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f839d8-24b5-461b-81cd-f6322a90c442 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:20.736253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f839d8-24b5-461b-81cd-f6322a90c442 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:20.736784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f839d8-24b5-461b-81cd-f6322a90c442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:20.738271 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f839d8-24b5-461b-81cd-f6322a90c442 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:20.749156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 187/369] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:20.749831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f839d8-24b5-461b-81cd-f6322a90c442 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:20.762878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:20.784704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae9c28-18ba-4028-b22d-55a12f921d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:20.785888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae9c28-18ba-4028-b22d-55a12f921d2f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:20.786174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae9c28-18ba-4028-b22d-55a12f921d2f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:20.786454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae9c28-18ba-4028-b22d-55a12f921d2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:20.812357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae9c28-18ba-4028-b22d-55a12f921d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-ebae9c28-18ba-4028-b22d-55a12f921d2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:20.825784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:20.826782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:20.827662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 183/370] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:40:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:20.828422 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae9c28-18ba-4028-b22d-55a12f921d2f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:20.903453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f839d8-24b5-461b-81cd-f6322a90c442 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:20.943749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05ff88-dd3a-4482-b392-0aa90d2abc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:20.944457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ff88-dd3a-4482-b392-0aa90d2abc15 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:20.944747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ff88-dd3a-4482-b392-0aa90d2abc15 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:20.945025 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05ff88-dd3a-4482-b392-0aa90d2abc15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:20.945868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05ff88-dd3a-4482-b392-0aa90d2abc15 None admin] RBAC: Adding query filter params (group.id=bf1c6135868f438e9f7f34737cf60e53, role.id=0f8e73b285204863a74f8af4ff7bbc86, scope.project.id=4060784cbaea460b93f362ae4cf23508) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:20.946184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05ff88-dd3a-4482-b392-0aa90d2abc15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:20.947637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05ff88-dd3a-4482-b392-0aa90d2abc15 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:20.954618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:20.960514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 188/371] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jul 19 20:40:20 2022] GET /identity/v3/role_assignments?group.id=bf1c6135868f438e9f7f34737cf60e53&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:20.961531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05ff88-dd3a-4482-b392-0aa90d2abc15 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:20.982404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae9c28-18ba-4028-b22d-55a12f921d2f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:21.032232 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9591c8-4aaf-4365-81a2-deb5f257c7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:21.033921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9591c8-4aaf-4365-81a2-deb5f257c7d9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:21.034720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9591c8-4aaf-4365-81a2-deb5f257c7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:21.035234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9591c8-4aaf-4365-81a2-deb5f257c7d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:21.059744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:21.097373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05ff88-dd3a-4482-b392-0aa90d2abc15 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:21.120942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9591c8-4aaf-4365-81a2-deb5f257c7d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:21.122939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9591c8-4aaf-4365-81a2-deb5f257c7d9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:21.135806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20a3c3-7caf-4b10-8b9a-9e2ef6795861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:21.136543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20a3c3-7caf-4b10-8b9a-9e2ef6795861 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:21.136835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20a3c3-7caf-4b10-8b9a-9e2ef6795861 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:21.137182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20a3c3-7caf-4b10-8b9a-9e2ef6795861 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:21.137950 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20a3c3-7caf-4b10-8b9a-9e2ef6795861 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:21.138308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20a3c3-7caf-4b10-8b9a-9e2ef6795861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:21.139787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20a3c3-7caf-4b10-8b9a-9e2ef6795861 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:21.150482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 189/372] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:21.156587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20a3c3-7caf-4b10-8b9a-9e2ef6795861 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:21.223784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:21.254007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 184/373] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:21.260438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9591c8-4aaf-4365-81a2-deb5f257c7d9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:21.275069 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20a3c3-7caf-4b10-8b9a-9e2ef6795861 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:21.319896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa283dba-0028-4fe4-ab9d-e325c33d1fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:21.320658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa283dba-0028-4fe4-ab9d-e325c33d1fd9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:21.320943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa283dba-0028-4fe4-ab9d-e325c33d1fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:21.321222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa283dba-0028-4fe4-ab9d-e325c33d1fd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:21.324307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:21.376230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9591c8-4aaf-4365-81a2-deb5f257c7d9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:21.399167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa283dba-0028-4fe4-ab9d-e325c33d1fd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:21.400687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa283dba-0028-4fe4-ab9d-e325c33d1fd9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:21.415559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1255f3-9c78-4f63-ac13-8740d00fbf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:21.416668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1255f3-9c78-4f63-ac13-8740d00fbf13 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:21.417078 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1255f3-9c78-4f63-ac13-8740d00fbf13 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:21.417532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1255f3-9c78-4f63-ac13-8740d00fbf13 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:21.419103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1255f3-9c78-4f63-ac13-8740d00fbf13 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:21.421068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1255f3-9c78-4f63-ac13-8740d00fbf13 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:21.423248 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 185/374] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:21 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:21.429257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1255f3-9c78-4f63-ac13-8740d00fbf13 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:21.483722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:21.509414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 190/375] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:21.517545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa283dba-0028-4fe4-ab9d-e325c33d1fd9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:21.542081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1255f3-9c78-4f63-ac13-8740d00fbf13 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:21.580725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:21.589245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bb695-99e7-4a3c-9116-a753c0352366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:21.590596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bb695-99e7-4a3c-9116-a753c0352366 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:21.591001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bb695-99e7-4a3c-9116-a753c0352366 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:21.591421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bb695-99e7-4a3c-9116-a753c0352366 None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:21.593015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5bb695-99e7-4a3c-9116-a753c0352366 None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:21.594732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5bb695-99e7-4a3c-9116-a753c0352366 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:21.597259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 186/376] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:21 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:21.605144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bb695-99e7-4a3c-9116-a753c0352366 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:21.672639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa283dba-0028-4fe4-ab9d-e325c33d1fd9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:21.722188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb6b4d-d77b-4cba-956a-766b975de67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:21.723075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6b4d-d77b-4cba-956a-766b975de67b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:21.723355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6b4d-d77b-4cba-956a-766b975de67b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:21.723634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6b4d-d77b-4cba-956a-766b975de67b None admin] PATH_INFO: `/v3/projects/3f322b7a5256429ea77db2c08c676fad` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:21.730935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb6b4d-d77b-4cba-956a-766b975de67b None admin] RBAC: Authorizing `identity:get_project(project_id=3f322b7a5256429ea77db2c08c676fad)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:21.732457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb6b4d-d77b-4cba-956a-766b975de67b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:21.737947 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 191/377] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:21 2022] GET /identity/v3/projects/3f322b7a5256429ea77db2c08c676fad => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:21.745042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb6b4d-d77b-4cba-956a-766b975de67b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:21.812279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bb695-99e7-4a3c-9116-a753c0352366 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:21.847664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:21.884641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81c903-eec0-46ec-8b58-dbcea30d1ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:21.885315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81c903-eec0-46ec-8b58-dbcea30d1ac9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:21.885594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81c903-eec0-46ec-8b58-dbcea30d1ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:21.885867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81c903-eec0-46ec-8b58-dbcea30d1ac9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:21.886698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81c903-eec0-46ec-8b58-dbcea30d1ac9 None admin] RBAC: Adding query filter params (group.id=bf1c6135868f438e9f7f34737cf60e53, role.id=aab847245001401cb4d3d43238298f07, scope.project.id=4060784cbaea460b93f362ae4cf23508) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:21.887008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81c903-eec0-46ec-8b58-dbcea30d1ac9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:21.888475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81c903-eec0-46ec-8b58-dbcea30d1ac9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:21.906078 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 187/378] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jul 19 20:40:21 2022] GET /identity/v3/role_assignments?group.id=bf1c6135868f438e9f7f34737cf60e53&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=aab847245001401cb4d3d43238298f07 => generated 260 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:21.936957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb6b4d-d77b-4cba-956a-766b975de67b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:21.940776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:21.985501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cc8b7-ba1f-4a3c-b55a-57821d187c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:21.986388 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cc8b7-ba1f-4a3c-b55a-57821d187c8a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:21.986669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cc8b7-ba1f-4a3c-b55a-57821d187c8a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:21.986947 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cc8b7-ba1f-4a3c-b55a-57821d187c8a None admin] PATH_INFO: `/v3/groups/4716fbf7c6954991b6fff25e6a7af528` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:21.996734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cc8b7-ba1f-4a3c-b55a-57821d187c8a None admin] RBAC: Authorizing `identity:get_group(group_id=4716fbf7c6954991b6fff25e6a7af528)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:21.998225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cc8b7-ba1f-4a3c-b55a-57821d187c8a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:22.000368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 192/379] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:21 2022] GET /identity/v3/groups/4716fbf7c6954991b6fff25e6a7af528 => generated 229 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:22.006318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81c903-eec0-46ec-8b58-dbcea30d1ac9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:22.171304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81c903-eec0-46ec-8b58-dbcea30d1ac9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:22.220013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6199d7b-8a95-4e43-9f6d-db2281836bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:22.220710 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6199d7b-8a95-4e43-9f6d-db2281836bca None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:22.221049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6199d7b-8a95-4e43-9f6d-db2281836bca None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:22.221328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6199d7b-8a95-4e43-9f6d-db2281836bca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:22.222164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6199d7b-8a95-4e43-9f6d-db2281836bca None admin] RBAC: Adding query filter params (group.id=4716fbf7c6954991b6fff25e6a7af528, role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=3f322b7a5256429ea77db2c08c676fad) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:22.222473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6199d7b-8a95-4e43-9f6d-db2281836bca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:22.224484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6199d7b-8a95-4e43-9f6d-db2281836bca None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:22.237855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 188/380] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jul 19 20:40:22 2022] GET /identity/v3/role_assignments?group.id=4716fbf7c6954991b6fff25e6a7af528&scope.project.id=3f322b7a5256429ea77db2c08c676fad&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:22.408532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c81315-12a4-4541-8834-2197ead41421 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:22.408899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c81315-12a4-4541-8834-2197ead41421 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:22.409090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c81315-12a4-4541-8834-2197ead41421 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:22.410774 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 193/381] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:22.416283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6a2fb-7bb8-4016-a38c-91e0f81b1b53 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:22.416605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6a2fb-7bb8-4016-a38c-91e0f81b1b53 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:22.417115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6a2fb-7bb8-4016-a38c-91e0f81b1b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:22.438021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-69b6a2fb-7bb8-4016-a38c-91e0f81b1b53 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:22.439441 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-69b6a2fb-7bb8-4016-a38c-91e0f81b1b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:22.494110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:22.543617 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 189/382] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:22.579161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c81315-12a4-4541-8834-2197ead41421 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:22.734412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c81315-12a4-4541-8834-2197ead41421 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:22.782999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811152d-1479-4712-a1e4-2bde9e3a328a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:22.783826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811152d-1479-4712-a1e4-2bde9e3a328a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:22.784113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811152d-1479-4712-a1e4-2bde9e3a328a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:22.784414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811152d-1479-4712-a1e4-2bde9e3a328a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:22.802761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811152d-1479-4712-a1e4-2bde9e3a328a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-5811152d-1479-4712-a1e4-2bde9e3a328a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:22.811819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:22.812836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:22.813641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:22.818352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 194/383] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:40:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:22.824477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6a2fb-7bb8-4016-a38c-91e0f81b1b53 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:22.952255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6a2fb-7bb8-4016-a38c-91e0f81b1b53 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:22.988965 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149d232-74b7-408c-9ccb-6474ce839bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:22.990089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149d232-74b7-408c-9ccb-6474ce839bb0 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:22.990371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149d232-74b7-408c-9ccb-6474ce839bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:22.990645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149d232-74b7-408c-9ccb-6474ce839bb0 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:22.991389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149d232-74b7-408c-9ccb-6474ce839bb0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:22.991691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149d232-74b7-408c-9ccb-6474ce839bb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:22.993096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149d232-74b7-408c-9ccb-6474ce839bb0 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:23.002442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 190/384] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:23.010077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811152d-1479-4712-a1e4-2bde9e3a328a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:23.056320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:23.136641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811152d-1479-4712-a1e4-2bde9e3a328a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:23.175253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8cd04-94d9-4209-845b-b209a08a64e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:23.176124 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8cd04-94d9-4209-845b-b209a08a64e0 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:23.176433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8cd04-94d9-4209-845b-b209a08a64e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:23.176706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8cd04-94d9-4209-845b-b209a08a64e0 None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:23.178770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8cd04-94d9-4209-845b-b209a08a64e0 None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:23.181002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8cd04-94d9-4209-845b-b209a08a64e0 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:23.184516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 195/385] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:23 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:23.193730 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149d232-74b7-408c-9ccb-6474ce839bb0 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:23.257705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:23.312352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97eccf5-f98c-45ac-bc4a-07b4c0a89860 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:23.312352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97eccf5-f98c-45ac-bc4a-07b4c0a89860 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:23.312352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97eccf5-f98c-45ac-bc4a-07b4c0a89860 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:23.313341 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 196/386] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:23.326244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0337cb-a5fb-4422-a66c-1821061cb98f None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:23.326532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0337cb-a5fb-4422-a66c-1821061cb98f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:23.326809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0337cb-a5fb-4422-a66c-1821061cb98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:23.335198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149d232-74b7-408c-9ccb-6474ce839bb0 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:23.366843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-fd0337cb-a5fb-4422-a66c-1821061cb98f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:23.368097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fd0337cb-a5fb-4422-a66c-1821061cb98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:23.373266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760b11d-58d1-45e1-b524-030b818e952c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:23.374135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760b11d-58d1-45e1-b524-030b818e952c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:23.374416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760b11d-58d1-45e1-b524-030b818e952c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:23.374686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760b11d-58d1-45e1-b524-030b818e952c None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:23.376736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760b11d-58d1-45e1-b524-030b818e952c None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:23.378180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760b11d-58d1-45e1-b524-030b818e952c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:23.380208 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 191/387] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:23 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:23.386445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760b11d-58d1-45e1-b524-030b818e952c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:23.511446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 197/388] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:23.549180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0337cb-a5fb-4422-a66c-1821061cb98f None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:23.580705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:23.582566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760b11d-58d1-45e1-b524-030b818e952c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:23.588881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:23.651428 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223dabb-ef95-4fd3-b6fd-1c8a9c574e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:23.652338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223dabb-ef95-4fd3-b6fd-1c8a9c574e3d None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:23.652626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223dabb-ef95-4fd3-b6fd-1c8a9c574e3d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:23.652899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223dabb-ef95-4fd3-b6fd-1c8a9c574e3d None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963/groups/bf1c6135868f438e9f7f34737cf60e53/roles/aab847245001401cb4d3d43238298f07` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:23.656714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223dabb-ef95-4fd3-b6fd-1c8a9c574e3d None admin] RBAC: Authorizing `identity:create_grant(project_id=e1b269e3c66347e1aa9595854069b963, group_id=bf1c6135868f438e9f7f34737cf60e53, role_id=aab847245001401cb4d3d43238298f07)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:23.660699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223dabb-ef95-4fd3-b6fd-1c8a9c574e3d None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:23.684762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 192/389] 10.222.0.85 () {58 vars in 1442 bytes} [Tue Jul 19 20:40:23 2022] PUT /identity/v3/projects/e1b269e3c66347e1aa9595854069b963/groups/bf1c6135868f438e9f7f34737cf60e53/roles/aab847245001401cb4d3d43238298f07 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:40:23.723616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0337cb-a5fb-4422-a66c-1821061cb98f None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:23.784562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b7a7c-cd31-4f85-a91a-1e04449326a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:23.785453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b7a7c-cd31-4f85-a91a-1e04449326a2 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:23.785737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b7a7c-cd31-4f85-a91a-1e04449326a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:23.786011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b7a7c-cd31-4f85-a91a-1e04449326a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:23.794151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b09d10-8f23-4cf0-986a-467e4f2a0678 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:23.794436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b09d10-8f23-4cf0-986a-467e4f2a0678 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:23.794711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b09d10-8f23-4cf0-986a-467e4f2a0678 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:23.796511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 193/390] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:23.808379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109916ad-b9dc-4478-8ca0-4f4141d50740 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:23.808666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109916ad-b9dc-4478-8ca0-4f4141d50740 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:23.808945 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109916ad-b9dc-4478-8ca0-4f4141d50740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:23.810676 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b7a7c-cd31-4f85-a91a-1e04449326a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-f90b7a7c-cd31-4f85-a91a-1e04449326a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:23.819661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:23.820852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:23.821722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:23.827885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 198/391] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:23.830905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b7a7c-cd31-4f85-a91a-1e04449326a2 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:23.838363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-109916ad-b9dc-4478-8ca0-4f4141d50740 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:23.839656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-109916ad-b9dc-4478-8ca0-4f4141d50740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:23.928747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 194/392] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 19 20:40:23.947541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:23.952796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109916ad-b9dc-4478-8ca0-4f4141d50740 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:23.961024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b7a7c-cd31-4f85-a91a-1e04449326a2 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:24.002745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b40074-0e7a-4a0c-a020-a53aac132d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:24.003613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b40074-0e7a-4a0c-a020-a53aac132d53 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:24.003995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b40074-0e7a-4a0c-a020-a53aac132d53 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:24.004286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b40074-0e7a-4a0c-a020-a53aac132d53 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:24.005230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b40074-0e7a-4a0c-a020-a53aac132d53 None admin] RBAC: Adding query filter params (name=member) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:24.005553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b40074-0e7a-4a0c-a020-a53aac132d53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:24.006948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b40074-0e7a-4a0c-a020-a53aac132d53 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:24.016706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 199/393] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:24.026819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b40074-0e7a-4a0c-a020-a53aac132d53 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:24.078116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:24.086800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109916ad-b9dc-4478-8ca0-4f4141d50740 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:24.158608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55405fcf-07e8-428e-94ae-82e8dbe687ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:24.159425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55405fcf-07e8-428e-94ae-82e8dbe687ed None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:24.159707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55405fcf-07e8-428e-94ae-82e8dbe687ed None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:24.159985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55405fcf-07e8-428e-94ae-82e8dbe687ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:24.177638 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b40074-0e7a-4a0c-a020-a53aac132d53 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:24.189882 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55405fcf-07e8-428e-94ae-82e8dbe687ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-55405fcf-07e8-428e-94ae-82e8dbe687ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:24.197805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:24.198705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:24.199531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:24.208860 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 195/394] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:24.209838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55405fcf-07e8-428e-94ae-82e8dbe687ed None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:24.239717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab089797-f4fd-41c7-8f07-baf509d20393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:24.240635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab089797-f4fd-41c7-8f07-baf509d20393 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:24.240925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab089797-f4fd-41c7-8f07-baf509d20393 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:24.241206 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab089797-f4fd-41c7-8f07-baf509d20393 None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:24.242458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab089797-f4fd-41c7-8f07-baf509d20393 None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:24.243960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab089797-f4fd-41c7-8f07-baf509d20393 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:24.245831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 200/395] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:24 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:24.251157 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab089797-f4fd-41c7-8f07-baf509d20393 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:24.271612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:24.361209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55405fcf-07e8-428e-94ae-82e8dbe687ed None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:24.395421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73484c55-4c42-4460-944c-fce47983e6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:24.396648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73484c55-4c42-4460-944c-fce47983e6ea None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:24.396930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73484c55-4c42-4460-944c-fce47983e6ea None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:24.397204 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73484c55-4c42-4460-944c-fce47983e6ea None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:24.397946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73484c55-4c42-4460-944c-fce47983e6ea None admin] RBAC: Adding query filter params (name=member) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:24.398250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73484c55-4c42-4460-944c-fce47983e6ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:24.399639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73484c55-4c42-4460-944c-fce47983e6ea None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:24.408489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 196/396] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:24.414760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73484c55-4c42-4460-944c-fce47983e6ea None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:24.451274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:24.487258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab089797-f4fd-41c7-8f07-baf509d20393 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:24.544302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c3d05-01bf-45e6-a5c7-b12b653d3c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:24.545211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c3d05-01bf-45e6-a5c7-b12b653d3c58 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:24.545488 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c3d05-01bf-45e6-a5c7-b12b653d3c58 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:24.545772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c3d05-01bf-45e6-a5c7-b12b653d3c58 None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:24.546983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c3d05-01bf-45e6-a5c7-b12b653d3c58 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:24.548515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c3d05-01bf-45e6-a5c7-b12b653d3c58 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:24.552645 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 201/397] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:24 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:24.561317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c3d05-01bf-45e6-a5c7-b12b653d3c58 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:24.608343 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73484c55-4c42-4460-944c-fce47983e6ea None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:24.667080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d19117-11b1-4f6e-b6f3-329af9229725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:24.668047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d19117-11b1-4f6e-b6f3-329af9229725 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:24.668347 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d19117-11b1-4f6e-b6f3-329af9229725 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:24.668935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d19117-11b1-4f6e-b6f3-329af9229725 None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:24.670646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d19117-11b1-4f6e-b6f3-329af9229725 None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:24.672476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d19117-11b1-4f6e-b6f3-329af9229725 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:24.674557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 197/398] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:24 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:24.675802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243f3e6-20ae-4d93-b365-9979df8e3782 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:24.676083 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243f3e6-20ae-4d93-b365-9979df8e3782 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:24.676387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243f3e6-20ae-4d93-b365-9979df8e3782 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:24.682319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 198/399] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:24.683947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf59500-d979-4e36-af9e-61380f89816a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:24.684237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf59500-d979-4e36-af9e-61380f89816a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:24.684555 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf59500-d979-4e36-af9e-61380f89816a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:24.704285 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:24.709151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-9bf59500-d979-4e36-af9e-61380f89816a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:24.710351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-9bf59500-d979-4e36-af9e-61380f89816a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:24.726635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c3d05-01bf-45e6-a5c7-b12b653d3c58 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:24.765536 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a85db-2da6-4b92-811d-8a3e33f34470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:24.766437 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a85db-2da6-4b92-811d-8a3e33f34470 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:24.766723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a85db-2da6-4b92-811d-8a3e33f34470 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:24.767005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a85db-2da6-4b92-811d-8a3e33f34470 None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963/groups/bf1c6135868f438e9f7f34737cf60e53/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:24.769028 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a85db-2da6-4b92-811d-8a3e33f34470 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1b269e3c66347e1aa9595854069b963, group_id=bf1c6135868f438e9f7f34737cf60e53, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:24.770584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a85db-2da6-4b92-811d-8a3e33f34470 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:24.787456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 202/400] 10.222.0.85 () {58 vars in 1442 bytes} [Tue Jul 19 20:40:24 2022] PUT /identity/v3/projects/e1b269e3c66347e1aa9595854069b963/groups/bf1c6135868f438e9f7f34737cf60e53/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:40:24.788625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a85db-2da6-4b92-811d-8a3e33f34470 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:24.792423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 199/401] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:24.818196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf59500-d979-4e36-af9e-61380f89816a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:24.842674 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:24.959548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a85db-2da6-4b92-811d-8a3e33f34470 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:25.008997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b4b30-f65c-413e-ab5e-5934a82af8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:25.009936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b4b30-f65c-413e-ab5e-5934a82af8d9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.010244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b4b30-f65c-413e-ab5e-5934a82af8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.010527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b4b30-f65c-413e-ab5e-5934a82af8d9 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.012260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b4b30-f65c-413e-ab5e-5934a82af8d9 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:25.014265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b4b30-f65c-413e-ab5e-5934a82af8d9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:25.021259 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 203/402] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:24 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:25.024137 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b4b30-f65c-413e-ab5e-5934a82af8d9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:25.043514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:25.060912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf59500-d979-4e36-af9e-61380f89816a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:25.131272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4778656-ea3c-4065-9c3e-2771ac2aaf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:25.132550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4778656-ea3c-4065-9c3e-2771ac2aaf76 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.134387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4778656-ea3c-4065-9c3e-2771ac2aaf76 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.136343 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4778656-ea3c-4065-9c3e-2771ac2aaf76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.161524 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4778656-ea3c-4065-9c3e-2771ac2aaf76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-a4778656-ea3c-4065-9c3e-2771ac2aaf76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:25.171669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:25.172957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:25.173767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:25.186110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 200/403] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:40:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:25.192616 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4778656-ea3c-4065-9c3e-2771ac2aaf76 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:25.215398 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b4b30-f65c-413e-ab5e-5934a82af8d9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:25.254019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd0fd3-ed4c-4b67-a808-688504276de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:25.255652 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd0fd3-ed4c-4b67-a808-688504276de7 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.255948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd0fd3-ed4c-4b67-a808-688504276de7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.256227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd0fd3-ed4c-4b67-a808-688504276de7 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508/groups/bf1c6135868f438e9f7f34737cf60e53/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.258328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd0fd3-ed4c-4b67-a808-688504276de7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4060784cbaea460b93f362ae4cf23508, group_id=bf1c6135868f438e9f7f34737cf60e53, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:25.259778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd0fd3-ed4c-4b67-a808-688504276de7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:25.279479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 204/404] 10.222.0.85 () {58 vars in 1442 bytes} [Tue Jul 19 20:40:25 2022] PUT /identity/v3/projects/4060784cbaea460b93f362ae4cf23508/groups/bf1c6135868f438e9f7f34737cf60e53/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:40:25.293099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dac14-7952-4c6c-9efe-72f2ef09d32c None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.293411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dac14-7952-4c6c-9efe-72f2ef09d32c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.293731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dac14-7952-4c6c-9efe-72f2ef09d32c None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.299411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 205/405] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:25.310337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b751e-ee78-4a7e-a67c-d9a78be025bc None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.310718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b751e-ee78-4a7e-a67c-d9a78be025bc None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.310942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b751e-ee78-4a7e-a67c-d9a78be025bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.346502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-b94b751e-ee78-4a7e-a67c-d9a78be025bc None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:25.346502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b94b751e-ee78-4a7e-a67c-d9a78be025bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:25.373344 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:25.450634 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4778656-ea3c-4065-9c3e-2771ac2aaf76 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:25.450957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:25.461161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 206/406] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:40:25.494844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b751e-ee78-4a7e-a67c-d9a78be025bc None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:25.495457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cd11a-3c9d-454e-9db0-f9fc2b2da7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:25.496164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cd11a-3c9d-454e-9db0-f9fc2b2da7b2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.496485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cd11a-3c9d-454e-9db0-f9fc2b2da7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.496765 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cd11a-3c9d-454e-9db0-f9fc2b2da7b2 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.497528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cd11a-3c9d-454e-9db0-f9fc2b2da7b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:25.497840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cd11a-3c9d-454e-9db0-f9fc2b2da7b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:25.499227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cd11a-3c9d-454e-9db0-f9fc2b2da7b2 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:25.509125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 201/407] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:25.517591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cd11a-3c9d-454e-9db0-f9fc2b2da7b2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:25.618256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b751e-ee78-4a7e-a67c-d9a78be025bc None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:25.635400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cd11a-3c9d-454e-9db0-f9fc2b2da7b2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:25.662755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a2122-bb10-485a-ad07-eac44412b008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:25.664068 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a2122-bb10-485a-ad07-eac44412b008 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.664580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a2122-bb10-485a-ad07-eac44412b008 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.665045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a2122-bb10-485a-ad07-eac44412b008 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.675573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a73b78-7d99-4540-b23b-9873b3ba58dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:25.676518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a73b78-7d99-4540-b23b-9873b3ba58dd None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.676804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a73b78-7d99-4540-b23b-9873b3ba58dd None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.677604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a73b78-7d99-4540-b23b-9873b3ba58dd None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.678833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a73b78-7d99-4540-b23b-9873b3ba58dd None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:25.680379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a73b78-7d99-4540-b23b-9873b3ba58dd None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:25.681415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a2122-bb10-485a-ad07-eac44412b008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:25.682509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 202/408] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:25 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:25.687877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a73b78-7d99-4540-b23b-9873b3ba58dd None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-2a0a2122-bb10-485a-ad07-eac44412b008 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:25.690902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:25.691797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:25.693178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:25.693785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:25.695872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 207/409] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:40:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:25.705775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a2122-bb10-485a-ad07-eac44412b008 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:25.798633 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a73b78-7d99-4540-b23b-9873b3ba58dd None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:25.811419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:25.851382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3f74e-73a8-4575-8151-336fbd95ca01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:25.852495 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3f74e-73a8-4575-8151-336fbd95ca01 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.852789 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3f74e-73a8-4575-8151-336fbd95ca01 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.853065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3f74e-73a8-4575-8151-336fbd95ca01 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.861288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3f74e-73a8-4575-8151-336fbd95ca01 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:25.862754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3f74e-73a8-4575-8151-336fbd95ca01 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:25.872913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 203/410] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:25 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:25.882085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3f74e-73a8-4575-8151-336fbd95ca01 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:25.900786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a2122-bb10-485a-ad07-eac44412b008 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:25.976725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce294d-f8a4-432a-b586-774bede0346d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:25.977446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce294d-f8a4-432a-b586-774bede0346d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:25.977741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce294d-f8a4-432a-b586-774bede0346d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:25.978019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce294d-f8a4-432a-b586-774bede0346d None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:25.978820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce294d-f8a4-432a-b586-774bede0346d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:25.979332 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce294d-f8a4-432a-b586-774bede0346d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:25.980588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce294d-f8a4-432a-b586-774bede0346d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:25.999677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 208/411] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:26.000453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:26.012347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce294d-f8a4-432a-b586-774bede0346d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:26.085793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3f74e-73a8-4575-8151-336fbd95ca01 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:26.128221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce294d-f8a4-432a-b586-774bede0346d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:26.150357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c832070-9fc0-42c7-9453-f35c6216693d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:26.151337 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832070-9fc0-42c7-9453-f35c6216693d None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:26.151618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832070-9fc0-42c7-9453-f35c6216693d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:26.151892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832070-9fc0-42c7-9453-f35c6216693d None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508/groups/bf1c6135868f438e9f7f34737cf60e53/roles/aab847245001401cb4d3d43238298f07` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:26.152605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:26.158148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c832070-9fc0-42c7-9453-f35c6216693d None admin] RBAC: Authorizing `identity:create_grant(project_id=4060784cbaea460b93f362ae4cf23508, group_id=bf1c6135868f438e9f7f34737cf60e53, role_id=aab847245001401cb4d3d43238298f07)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:26.159600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c832070-9fc0-42c7-9453-f35c6216693d None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:26.166989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ef5de-0027-4cb9-bc15-48ec6523b31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:26.167947 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ef5de-0027-4cb9-bc15-48ec6523b31f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:26.168357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ef5de-0027-4cb9-bc15-48ec6523b31f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:26.168678 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ef5de-0027-4cb9-bc15-48ec6523b31f None admin] PATH_INFO: `/v3/groups/4716fbf7c6954991b6fff25e6a7af528` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:26.170147 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ef5de-0027-4cb9-bc15-48ec6523b31f None admin] RBAC: Authorizing `identity:get_group(group_id=4716fbf7c6954991b6fff25e6a7af528)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:26.171830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ef5de-0027-4cb9-bc15-48ec6523b31f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:26.174158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 209/412] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:26 2022] GET /identity/v3/groups/4716fbf7c6954991b6fff25e6a7af528 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:26.181212 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ef5de-0027-4cb9-bc15-48ec6523b31f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:26.184124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 204/413] 10.222.0.85 () {58 vars in 1442 bytes} [Tue Jul 19 20:40:25 2022] PUT /identity/v3/projects/4060784cbaea460b93f362ae4cf23508/groups/bf1c6135868f438e9f7f34737cf60e53/roles/aab847245001401cb4d3d43238298f07 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:40:26.238529 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda207b0-38b9-4535-8a84-ad3ef4bffa99 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:26.238852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda207b0-38b9-4535-8a84-ad3ef4bffa99 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:26.239096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda207b0-38b9-4535-8a84-ad3ef4bffa99 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:26.241369 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 205/414] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:40:26.246824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0450389-851c-4f16-bdd7-c866268ea42c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:26.247112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0450389-851c-4f16-bdd7-c866268ea42c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:26.247377 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0450389-851c-4f16-bdd7-c866268ea42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:26.265437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-f0450389-851c-4f16-bdd7-c866268ea42c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:26.266722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f0450389-851c-4f16-bdd7-c866268ea42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:26.365481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 206/415] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:40:26.395394 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ef5de-0027-4cb9-bc15-48ec6523b31f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:26.397611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0450389-851c-4f16-bdd7-c866268ea42c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:26.439514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8fa80-e612-4964-bb98-674a8b4853fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:26.440469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8fa80-e612-4964-bb98-674a8b4853fe None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:26.440707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8fa80-e612-4964-bb98-674a8b4853fe None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:26.440913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8fa80-e612-4964-bb98-674a8b4853fe None admin] PATH_INFO: `/v3/projects/3f322b7a5256429ea77db2c08c676fad` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:26.443227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8fa80-e612-4964-bb98-674a8b4853fe None admin] RBAC: Authorizing `identity:get_project(project_id=3f322b7a5256429ea77db2c08c676fad)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:26.444975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8fa80-e612-4964-bb98-674a8b4853fe None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:26.447113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 210/416] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:26 2022] GET /identity/v3/projects/3f322b7a5256429ea77db2c08c676fad => generated 344 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:26.454392 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8fa80-e612-4964-bb98-674a8b4853fe None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:26.478844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:26.523012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0450389-851c-4f16-bdd7-c866268ea42c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:26.540130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:26.562507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63b955-947d-4400-8d66-5042d629d996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:26.564129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63b955-947d-4400-8d66-5042d629d996 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:26.564676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63b955-947d-4400-8d66-5042d629d996 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:26.565149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63b955-947d-4400-8d66-5042d629d996 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:26.577355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63b955-947d-4400-8d66-5042d629d996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-ca63b955-947d-4400-8d66-5042d629d996 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:26.586578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:26.587494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:26.588309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:26.591137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 207/417] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:40:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:26.611831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8fa80-e612-4964-bb98-674a8b4853fe None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:26.613182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63b955-947d-4400-8d66-5042d629d996 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:26.663216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbc2df-0a4e-4abe-a9ac-87768ad4d293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:26.664057 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbc2df-0a4e-4abe-a9ac-87768ad4d293 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:26.664421 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbc2df-0a4e-4abe-a9ac-87768ad4d293 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:26.664703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbc2df-0a4e-4abe-a9ac-87768ad4d293 None admin] PATH_INFO: `/v3/projects/3f322b7a5256429ea77db2c08c676fad/groups/4716fbf7c6954991b6fff25e6a7af528/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:26.669349 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbc2df-0a4e-4abe-a9ac-87768ad4d293 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f322b7a5256429ea77db2c08c676fad, group_id=4716fbf7c6954991b6fff25e6a7af528, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:26.671504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbc2df-0a4e-4abe-a9ac-87768ad4d293 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:26.694293 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 211/418] 10.222.0.85 () {58 vars in 1442 bytes} [Tue Jul 19 20:40:26 2022] PUT /identity/v3/projects/3f322b7a5256429ea77db2c08c676fad/groups/4716fbf7c6954991b6fff25e6a7af528/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:40:26.800274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63b955-947d-4400-8d66-5042d629d996 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:26.847189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:26.870708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afae81d-e0e2-49d5-850d-182621d3fe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:26.871395 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afae81d-e0e2-49d5-850d-182621d3fe16 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:26.871675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afae81d-e0e2-49d5-850d-182621d3fe16 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:26.871948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afae81d-e0e2-49d5-850d-182621d3fe16 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:26.872739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afae81d-e0e2-49d5-850d-182621d3fe16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:26.873120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afae81d-e0e2-49d5-850d-182621d3fe16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:26.874559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afae81d-e0e2-49d5-850d-182621d3fe16 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:26.888930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 208/419] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:26.894254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbc2df-0a4e-4abe-a9ac-87768ad4d293 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:27.024211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbc2df-0a4e-4abe-a9ac-87768ad4d293 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:27.042906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:27.070717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f7b88-3cee-46de-b30b-d9f662856036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:27.071370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f7b88-3cee-46de-b30b-d9f662856036 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:27.071658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f7b88-3cee-46de-b30b-d9f662856036 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:27.071938 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f7b88-3cee-46de-b30b-d9f662856036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:27.151749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f7b88-3cee-46de-b30b-d9f662856036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:27.153243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f7b88-3cee-46de-b30b-d9f662856036 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:27.258530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 212/420] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:40:27.264105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afae81d-e0e2-49d5-850d-182621d3fe16 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:27.407181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afae81d-e0e2-49d5-850d-182621d3fe16 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:27.446459 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7643b42-47d0-49dc-b488-b5ee1a5d81a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:27.447331 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643b42-47d0-49dc-b488-b5ee1a5d81a7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:27.447611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643b42-47d0-49dc-b488-b5ee1a5d81a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:27.447896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643b42-47d0-49dc-b488-b5ee1a5d81a7 None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:27.449138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7643b42-47d0-49dc-b488-b5ee1a5d81a7 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:27.450594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7643b42-47d0-49dc-b488-b5ee1a5d81a7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:27.452628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 209/421] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:27 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:27.458621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f7b88-3cee-46de-b30b-d9f662856036 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:27.481358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:27.500156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e4028-b87a-413c-bd10-6c25b0fc2348 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:27.500501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e4028-b87a-413c-bd10-6c25b0fc2348 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:27.500738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e4028-b87a-413c-bd10-6c25b0fc2348 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:27.502168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 210/422] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:40:27.509658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6792bd-c40f-4086-91bd-d438591a02d9 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:27.509950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6792bd-c40f-4086-91bd-d438591a02d9 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:27.510212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6792bd-c40f-4086-91bd-d438591a02d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:27.536318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-cc6792bd-c40f-4086-91bd-d438591a02d9 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:27.537663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cc6792bd-c40f-4086-91bd-d438591a02d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:27.602754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f7b88-3cee-46de-b30b-d9f662856036 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:27.644879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:27.649124 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e0ea7-47dd-4760-b887-3e0885794f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:27.650327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e0ea7-47dd-4760-b887-3e0885794f1a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:27.650854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e0ea7-47dd-4760-b887-3e0885794f1a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:27.651337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e0ea7-47dd-4760-b887-3e0885794f1a None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:27.652865 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e0ea7-47dd-4760-b887-3e0885794f1a None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:27.654853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e0ea7-47dd-4760-b887-3e0885794f1a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:27.657360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 213/423] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:27 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:27.664781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e0ea7-47dd-4760-b887-3e0885794f1a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:27.695954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 211/424] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:40:27.748718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6792bd-c40f-4086-91bd-d438591a02d9 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:27.763711 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:27.798770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e0ea7-47dd-4760-b887-3e0885794f1a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:27.845044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ff7d1-98c2-4494-9292-ec8e89d1325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:27.847045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff7d1-98c2-4494-9292-ec8e89d1325c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:27.847437 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff7d1-98c2-4494-9292-ec8e89d1325c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:27.847814 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ff7d1-98c2-4494-9292-ec8e89d1325c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:27.849613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ff7d1-98c2-4494-9292-ec8e89d1325c None admin] RBAC: Adding query filter params (group.id=bf1c6135868f438e9f7f34737cf60e53, role.id=aab847245001401cb4d3d43238298f07, scope.project.id=e1b269e3c66347e1aa9595854069b963) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:27.850045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ff7d1-98c2-4494-9292-ec8e89d1325c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:27.853653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ff7d1-98c2-4494-9292-ec8e89d1325c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:27.872831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 214/425] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jul 19 20:40:27 2022] GET /identity/v3/role_assignments?group.id=bf1c6135868f438e9f7f34737cf60e53&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=aab847245001401cb4d3d43238298f07 => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:27.974866 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6792bd-c40f-4086-91bd-d438591a02d9 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:27.984307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f8fd4-f77e-4854-b677-156bfe7ed009 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:27.984871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f8fd4-f77e-4854-b677-156bfe7ed009 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:27.984950 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f8fd4-f77e-4854-b677-156bfe7ed009 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:27.989307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 215/426] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:40:27.996048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a7c37-2740-4844-b976-96d5a231e940 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:27.996345 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a7c37-2740-4844-b976-96d5a231e940 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:27.996647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a7c37-2740-4844-b976-96d5a231e940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:28.029456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-798a7c37-2740-4844-b976-96d5a231e940 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:28.031895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-798a7c37-2740-4844-b976-96d5a231e940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:28.033721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df4d29-4310-4d73-bfd7-9ca46e4ecc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:28.034810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df4d29-4310-4d73-bfd7-9ca46e4ecc45 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:28.035274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df4d29-4310-4d73-bfd7-9ca46e4ecc45 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:28.035727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df4d29-4310-4d73-bfd7-9ca46e4ecc45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:28.058703 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df4d29-4310-4d73-bfd7-9ca46e4ecc45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-68df4d29-4310-4d73-bfd7-9ca46e4ecc45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:28.067691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:28.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:28.069518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:28.072353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 212/427] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:28.079164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df4d29-4310-4d73-bfd7-9ca46e4ecc45 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:28.146124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:28.156784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 216/428] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:40:28.193475 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a7c37-2740-4844-b976-96d5a231e940 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:28.222138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df4d29-4310-4d73-bfd7-9ca46e4ecc45 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:28.272597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2aa4f-ed4d-42c6-80c9-dccbd66a2421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:28.273903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2aa4f-ed4d-42c6-80c9-dccbd66a2421 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:28.274354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2aa4f-ed4d-42c6-80c9-dccbd66a2421 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:28.274782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2aa4f-ed4d-42c6-80c9-dccbd66a2421 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:28.276629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2aa4f-ed4d-42c6-80c9-dccbd66a2421 None admin] RBAC: Adding query filter params (name=member) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:28.276916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2aa4f-ed4d-42c6-80c9-dccbd66a2421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:28.278551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2aa4f-ed4d-42c6-80c9-dccbd66a2421 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:28.289794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 213/429] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:28.297367 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2aa4f-ed4d-42c6-80c9-dccbd66a2421 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:28.321746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:28.348691 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a7c37-2740-4844-b976-96d5a231e940 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:28.394822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042408fe-f18c-4ea8-9b67-e434e98beaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:28.396699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042408fe-f18c-4ea8-9b67-e434e98beaf2 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:28.397253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042408fe-f18c-4ea8-9b67-e434e98beaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:28.397738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042408fe-f18c-4ea8-9b67-e434e98beaf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:28.411030 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042408fe-f18c-4ea8-9b67-e434e98beaf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-042408fe-f18c-4ea8-9b67-e434e98beaf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:28.420264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:28.421365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 20:40:28.422277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:28.422879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2aa4f-ed4d-42c6-80c9-dccbd66a2421 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:28.424101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:28.425470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 217/430] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:28.434488 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042408fe-f18c-4ea8-9b67-e434e98beaf2 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:28.460285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72df05d-a87c-49c3-9272-221e431c2c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:28.461020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72df05d-a87c-49c3-9272-221e431c2c65 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:28.461289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72df05d-a87c-49c3-9272-221e431c2c65 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:28.461569 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72df05d-a87c-49c3-9272-221e431c2c65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:28.535700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72df05d-a87c-49c3-9272-221e431c2c65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:28.537285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72df05d-a87c-49c3-9272-221e431c2c65 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:28.551688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042408fe-f18c-4ea8-9b67-e434e98beaf2 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:28.594400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3dfc3-f50c-4a50-a045-51b6888aa224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:28.595773 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3dfc3-f50c-4a50-a045-51b6888aa224 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:28.596206 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3dfc3-f50c-4a50-a045-51b6888aa224 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:28.596687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3dfc3-f50c-4a50-a045-51b6888aa224 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:28.597619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3dfc3-f50c-4a50-a045-51b6888aa224 None admin] RBAC: Adding query filter params (name=member) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:28.598080 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3dfc3-f50c-4a50-a045-51b6888aa224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:28.599827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3dfc3-f50c-4a50-a045-51b6888aa224 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:28.616025 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 218/431] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:40:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:28.617313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3dfc3-f50c-4a50-a045-51b6888aa224 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:28.620932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:28.650644 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 214/432] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:28.656019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72df05d-a87c-49c3-9272-221e431c2c65 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:28.661536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:28.741677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3dfc3-f50c-4a50-a045-51b6888aa224 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:28.774578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72df05d-a87c-49c3-9272-221e431c2c65 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:28.783290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34adf6-c6b6-4674-8f04-f7aac82e90e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:28.784403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34adf6-c6b6-4674-8f04-f7aac82e90e2 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:28.784872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34adf6-c6b6-4674-8f04-f7aac82e90e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:28.785341 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34adf6-c6b6-4674-8f04-f7aac82e90e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:28.814086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c454f5-b133-4f9e-8a1d-b334a530ba7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:28.815281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c454f5-b133-4f9e-8a1d-b334a530ba7a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:28.815764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c454f5-b133-4f9e-8a1d-b334a530ba7a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:28.816237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c454f5-b133-4f9e-8a1d-b334a530ba7a None admin] PATH_INFO: `/v3/projects/e1b269e3c66347e1aa9595854069b963` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:28.818271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c454f5-b133-4f9e-8a1d-b334a530ba7a None admin] RBAC: Authorizing `identity:get_project(project_id=e1b269e3c66347e1aa9595854069b963)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:28.819726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c454f5-b133-4f9e-8a1d-b334a530ba7a None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:28.826517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 215/433] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:28 2022] GET /identity/v3/projects/e1b269e3c66347e1aa9595854069b963 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:28.828339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c454f5-b133-4f9e-8a1d-b334a530ba7a None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:28.866488 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34adf6-c6b6-4674-8f04-f7aac82e90e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:28.868573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34adf6-c6b6-4674-8f04-f7aac82e90e2 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:28.870981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:28.913920 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:28.946412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c454f5-b133-4f9e-8a1d-b334a530ba7a None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:28.985046 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 219/434] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:28.991445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d629a9-8502-49da-b6a1-1ba99c49be6e None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:28.991445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d629a9-8502-49da-b6a1-1ba99c49be6e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:28.991445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d629a9-8502-49da-b6a1-1ba99c49be6e None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:28.991445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 220/435] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:28.994500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d629a9-8502-49da-b6a1-1ba99c49be6e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:29.001206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa4e2b-b112-4776-aaec-0b877aca6312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:29.002180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa4e2b-b112-4776-aaec-0b877aca6312 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.002570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa4e2b-b112-4776-aaec-0b877aca6312 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.002919 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa4e2b-b112-4776-aaec-0b877aca6312 None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.004278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa4e2b-b112-4776-aaec-0b877aca6312 None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:29.006340 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa4e2b-b112-4776-aaec-0b877aca6312 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:29.008053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 216/436] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:28 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:29.009338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ea9a0-ce96-4314-8463-b1617e025658 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.009629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ea9a0-ce96-4314-8463-b1617e025658 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.009980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ea9a0-ce96-4314-8463-b1617e025658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.035318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-048ea9a0-ce96-4314-8463-b1617e025658 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:29.037019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-048ea9a0-ce96-4314-8463-b1617e025658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:29.120448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 217/437] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:29.120448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ea9a0-ce96-4314-8463-b1617e025658 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:29.139913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d629a9-8502-49da-b6a1-1ba99c49be6e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:29.141696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:29.179269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279abb1-cc1b-4029-a09f-55f57a23ba06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:29.180439 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279abb1-cc1b-4029-a09f-55f57a23ba06 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.180888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279abb1-cc1b-4029-a09f-55f57a23ba06 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.181164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279abb1-cc1b-4029-a09f-55f57a23ba06 None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.182621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279abb1-cc1b-4029-a09f-55f57a23ba06 None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:29.184038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279abb1-cc1b-4029-a09f-55f57a23ba06 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:29.186340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 221/438] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:28 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:29.186975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279abb1-cc1b-4029-a09f-55f57a23ba06 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:29.190987 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:29.232467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ea9a0-ce96-4314-8463-b1617e025658 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:29.268291 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e29a2b-dfe1-4899-a2c0-171a2540c319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:29.269009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e29a2b-dfe1-4899-a2c0-171a2540c319 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.269291 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e29a2b-dfe1-4899-a2c0-171a2540c319 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.269562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e29a2b-dfe1-4899-a2c0-171a2540c319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.270388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e29a2b-dfe1-4899-a2c0-171a2540c319 None admin] RBAC: Adding query filter params (group.id=bf1c6135868f438e9f7f34737cf60e53, role.id=0f8e73b285204863a74f8af4ff7bbc86, scope.project.id=e1b269e3c66347e1aa9595854069b963) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:29.271259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e29a2b-dfe1-4899-a2c0-171a2540c319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:29.272717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e29a2b-dfe1-4899-a2c0-171a2540c319 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:29.279179 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 218/439] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jul 19 20:40:29 2022] GET /identity/v3/role_assignments?group.id=bf1c6135868f438e9f7f34737cf60e53&scope.project.id=e1b269e3c66347e1aa9595854069b963&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:29.279763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e29a2b-dfe1-4899-a2c0-171a2540c319 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:29.337933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279abb1-cc1b-4029-a09f-55f57a23ba06 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:29.394168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845a566-1ee8-4c69-8d80-e2bfe0758cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:29.394913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845a566-1ee8-4c69-8d80-e2bfe0758cd7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.395204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845a566-1ee8-4c69-8d80-e2bfe0758cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.395510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845a566-1ee8-4c69-8d80-e2bfe0758cd7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.407734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845a566-1ee8-4c69-8d80-e2bfe0758cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:29.410753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e29a2b-dfe1-4899-a2c0-171a2540c319 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-8845a566-1ee8-4c69-8d80-e2bfe0758cd7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:29.416662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:29.417622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:29.418435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 222/440] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:40:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:29.419352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b25bd1-c9f1-4d26-8a3e-d8258d5ecc6e None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.419644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b25bd1-c9f1-4d26-8a3e-d8258d5ecc6e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.419929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b25bd1-c9f1-4d26-8a3e-d8258d5ecc6e None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.421613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 223/441] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:29.424176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b25bd1-c9f1-4d26-8a3e-d8258d5ecc6e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:29.454315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc94d56-cd69-42e0-9f58-e4e133a123fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:29.455175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc94d56-cd69-42e0-9f58-e4e133a123fa None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.455460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc94d56-cd69-42e0-9f58-e4e133a123fa None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.455728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc94d56-cd69-42e0-9f58-e4e133a123fa None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.457875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc94d56-cd69-42e0-9f58-e4e133a123fa None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:29.459412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc94d56-cd69-42e0-9f58-e4e133a123fa None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:29.466350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 219/442] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:29 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:29.466350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6eddc-de75-4d84-ab92-ffb2a1ed2c67 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.466350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6eddc-de75-4d84-ab92-ffb2a1ed2c67 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.466350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6eddc-de75-4d84-ab92-ffb2a1ed2c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.469325 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:29.471320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:29.485290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3bd6eddc-de75-4d84-ab92-ffb2a1ed2c67 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:29.486747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3bd6eddc-de75-4d84-ab92-ffb2a1ed2c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:29.565805 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b25bd1-c9f1-4d26-8a3e-d8258d5ecc6e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:29.589255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 220/443] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:29.590539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6eddc-de75-4d84-ab92-ffb2a1ed2c67 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:29.619736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ca47e-1f65-4242-8ec1-cf830204810b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:29.620470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ca47e-1f65-4242-8ec1-cf830204810b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.620760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ca47e-1f65-4242-8ec1-cf830204810b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.621041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ca47e-1f65-4242-8ec1-cf830204810b None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.621803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ca47e-1f65-4242-8ec1-cf830204810b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:29.622117 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ca47e-1f65-4242-8ec1-cf830204810b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:29.623498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ca47e-1f65-4242-8ec1-cf830204810b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:29.634239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 224/444] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:29.635357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ca47e-1f65-4242-8ec1-cf830204810b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:29.746948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6eddc-de75-4d84-ab92-ffb2a1ed2c67 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:29.759832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:29.775505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ca47e-1f65-4242-8ec1-cf830204810b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:29.791573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:29.800565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50374f26-7808-4bcf-b0fe-f7ada372957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:29.801525 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50374f26-7808-4bcf-b0fe-f7ada372957f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.802558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50374f26-7808-4bcf-b0fe-f7ada372957f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.803037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50374f26-7808-4bcf-b0fe-f7ada372957f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.803883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50374f26-7808-4bcf-b0fe-f7ada372957f None admin] RBAC: Adding query filter params (group.id=bf1c6135868f438e9f7f34737cf60e53, role.id=0f8e73b285204863a74f8af4ff7bbc86, scope.project.id=4060784cbaea460b93f362ae4cf23508) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:29.804195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50374f26-7808-4bcf-b0fe-f7ada372957f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:29.805691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50374f26-7808-4bcf-b0fe-f7ada372957f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:29.813833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 221/445] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jul 19 20:40:29 2022] GET /identity/v3/role_assignments?group.id=bf1c6135868f438e9f7f34737cf60e53&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=0f8e73b285204863a74f8af4ff7bbc86 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:29.814485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50374f26-7808-4bcf-b0fe-f7ada372957f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:29.825305 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96958498-6f2f-4166-92c7-1368cda65fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:29.826309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96958498-6f2f-4166-92c7-1368cda65fe3 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:29.826600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96958498-6f2f-4166-92c7-1368cda65fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:29.826948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96958498-6f2f-4166-92c7-1368cda65fe3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:29.839745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96958498-6f2f-4166-92c7-1368cda65fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-96958498-6f2f-4166-92c7-1368cda65fe3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:29.847750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:29.849315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:29.850126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:29.851249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 225/446] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:40:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:29.857009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96958498-6f2f-4166-92c7-1368cda65fe3 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:29.965593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50374f26-7808-4bcf-b0fe-f7ada372957f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:30.012934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3badd18-3e29-475e-aaeb-d52560f4e14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:30.013524 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3badd18-3e29-475e-aaeb-d52560f4e14c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:30.013749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3badd18-3e29-475e-aaeb-d52560f4e14c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:30.016422 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3badd18-3e29-475e-aaeb-d52560f4e14c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:30.022732 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96958498-6f2f-4166-92c7-1368cda65fe3 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:30.074262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dbddc-4e0e-4567-bda1-3310f0029794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:30.075203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dbddc-4e0e-4567-bda1-3310f0029794 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:30.075646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dbddc-4e0e-4567-bda1-3310f0029794 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:30.075924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dbddc-4e0e-4567-bda1-3310f0029794 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:30.077099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dbddc-4e0e-4567-bda1-3310f0029794 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:30.077099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dbddc-4e0e-4567-bda1-3310f0029794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:30.079344 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6dbddc-4e0e-4567-bda1-3310f0029794 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:30.089095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 226/447] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:30.095331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dbddc-4e0e-4567-bda1-3310f0029794 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:30.101212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:30.108971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3badd18-3e29-475e-aaeb-d52560f4e14c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:30.110450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3badd18-3e29-475e-aaeb-d52560f4e14c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:30.120003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:30.246094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6dbddc-4e0e-4567-bda1-3310f0029794 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:30.251391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 222/448] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:30.257190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3badd18-3e29-475e-aaeb-d52560f4e14c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:30.292441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b8dc3-a43f-4163-b0c0-ded40c20cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:30.292994 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b8dc3-a43f-4163-b0c0-ded40c20cb29 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:30.293246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b8dc3-a43f-4163-b0c0-ded40c20cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:30.293528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b8dc3-a43f-4163-b0c0-ded40c20cb29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:30.360536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:30.366831 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3badd18-3e29-475e-aaeb-d52560f4e14c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:30.372524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b8dc3-a43f-4163-b0c0-ded40c20cb29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:30.374018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b8dc3-a43f-4163-b0c0-ded40c20cb29 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:30.390190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:30.410352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c10c7-d603-4a4b-ad9c-d40f1a76e31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:30.413013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c10c7-d603-4a4b-ad9c-d40f1a76e31f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:30.413702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c10c7-d603-4a4b-ad9c-d40f1a76e31f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:30.414224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c10c7-d603-4a4b-ad9c-d40f1a76e31f None admin] PATH_INFO: `/v3/projects/4060784cbaea460b93f362ae4cf23508` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:30.415587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c10c7-d603-4a4b-ad9c-d40f1a76e31f None admin] RBAC: Authorizing `identity:get_project(project_id=4060784cbaea460b93f362ae4cf23508)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:30.417084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c10c7-d603-4a4b-ad9c-d40f1a76e31f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:30.419021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 223/449] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:30 2022] GET /identity/v3/projects/4060784cbaea460b93f362ae4cf23508 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:30.424587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c10c7-d603-4a4b-ad9c-d40f1a76e31f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:30.494685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 227/450] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:40:30.500551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b8dc3-a43f-4163-b0c0-ded40c20cb29 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:30.560950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c10c7-d603-4a4b-ad9c-d40f1a76e31f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:30.598627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651309f-8b0a-41c5-9be1-5a9f54879673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:30.599484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651309f-8b0a-41c5-9be1-5a9f54879673 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:30.599776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651309f-8b0a-41c5-9be1-5a9f54879673 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:30.600056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651309f-8b0a-41c5-9be1-5a9f54879673 None admin] PATH_INFO: `/v3/groups/bf1c6135868f438e9f7f34737cf60e53` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:30.601293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651309f-8b0a-41c5-9be1-5a9f54879673 None admin] RBAC: Authorizing `identity:get_group(group_id=bf1c6135868f438e9f7f34737cf60e53)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:30.602754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651309f-8b0a-41c5-9be1-5a9f54879673 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:30.608376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 224/451] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:30 2022] GET /identity/v3/groups/bf1c6135868f438e9f7f34737cf60e53 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:30.611406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651309f-8b0a-41c5-9be1-5a9f54879673 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:30.612226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b8dc3-a43f-4163-b0c0-ded40c20cb29 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:30.627747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:30.655194 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc1790-519a-4f28-9a03-54162443ca06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:30.656060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc1790-519a-4f28-9a03-54162443ca06 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:30.656370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc1790-519a-4f28-9a03-54162443ca06 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:30.656703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc1790-519a-4f28-9a03-54162443ca06 None admin] PATH_INFO: `/v3/projects/3f322b7a5256429ea77db2c08c676fad` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:30.657328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:30.657911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc1790-519a-4f28-9a03-54162443ca06 None admin] RBAC: Authorizing `identity:get_project(project_id=3f322b7a5256429ea77db2c08c676fad)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:30.659708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc1790-519a-4f28-9a03-54162443ca06 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:30.661795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 228/452] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:40:30 2022] GET /identity/v3/projects/3f322b7a5256429ea77db2c08c676fad => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:30.667626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc1790-519a-4f28-9a03-54162443ca06 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:30.731753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651309f-8b0a-41c5-9be1-5a9f54879673 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:30.767259 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc1790-519a-4f28-9a03-54162443ca06 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:30.767606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee6487-97e8-417e-8289-cc1deea19ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:30.768294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee6487-97e8-417e-8289-cc1deea19ff3 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:30.768615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee6487-97e8-417e-8289-cc1deea19ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:30.768911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee6487-97e8-417e-8289-cc1deea19ff3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:30.769771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee6487-97e8-417e-8289-cc1deea19ff3 None admin] RBAC: Adding query filter params (group.id=bf1c6135868f438e9f7f34737cf60e53, role.id=aab847245001401cb4d3d43238298f07, scope.project.id=4060784cbaea460b93f362ae4cf23508) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:30.770104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee6487-97e8-417e-8289-cc1deea19ff3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:30.772137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee6487-97e8-417e-8289-cc1deea19ff3 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:30.778953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 225/453] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jul 19 20:40:30 2022] GET /identity/v3/role_assignments?group.id=bf1c6135868f438e9f7f34737cf60e53&scope.project.id=4060784cbaea460b93f362ae4cf23508&role.id=aab847245001401cb4d3d43238298f07 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:30.806353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fc906-0c2f-4042-a585-1abced5b72f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:30.807502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fc906-0c2f-4042-a585-1abced5b72f1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:30.807872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fc906-0c2f-4042-a585-1abced5b72f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:30.808193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fc906-0c2f-4042-a585-1abced5b72f1 None admin] PATH_INFO: `/v3/groups/4716fbf7c6954991b6fff25e6a7af528` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:30.809998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fc906-0c2f-4042-a585-1abced5b72f1 None admin] RBAC: Authorizing `identity:get_group(group_id=4716fbf7c6954991b6fff25e6a7af528)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:30.811558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fc906-0c2f-4042-a585-1abced5b72f1 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:30.813318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 229/454] 10.222.0.85 () {56 vars in 1180 bytes} [Tue Jul 19 20:40:30 2022] GET /identity/v3/groups/4716fbf7c6954991b6fff25e6a7af528 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:30.820416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee6487-97e8-417e-8289-cc1deea19ff3 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:30.929104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:30.935787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee6487-97e8-417e-8289-cc1deea19ff3 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:30.971182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cd1f7-8607-42ad-b21a-662c6e7472c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:30.971850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cd1f7-8607-42ad-b21a-662c6e7472c3 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:30.972134 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cd1f7-8607-42ad-b21a-662c6e7472c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:30.972439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cd1f7-8607-42ad-b21a-662c6e7472c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:30.973276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cd1f7-8607-42ad-b21a-662c6e7472c3 None admin] RBAC: Adding query filter params (group.id=4716fbf7c6954991b6fff25e6a7af528, role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=3f322b7a5256429ea77db2c08c676fad) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:30.973640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cd1f7-8607-42ad-b21a-662c6e7472c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:30.975054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cd1f7-8607-42ad-b21a-662c6e7472c3 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:30.981148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 226/455] 10.222.0.85 () {56 vars in 1376 bytes} [Tue Jul 19 20:40:30 2022] GET /identity/v3/role_assignments?group.id=4716fbf7c6954991b6fff25e6a7af528&scope.project.id=3f322b7a5256429ea77db2c08c676fad&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:33.747055 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0167ad-e44a-4c09-ab15-901bef1dee51 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:33.747598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0167ad-e44a-4c09-ab15-901bef1dee51 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:33.747598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0167ad-e44a-4c09-ab15-901bef1dee51 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:33.749292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 230/456] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:33.754305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d07ee5-df90-4788-9689-d7286319d716 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:33.754728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d07ee5-df90-4788-9689-d7286319d716 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:33.754929 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d07ee5-df90-4788-9689-d7286319d716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:33.781334 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d2d07ee5-df90-4788-9689-d7286319d716 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:33.782568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d2d07ee5-df90-4788-9689-d7286319d716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:33.885449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 227/457] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:33.920962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0167ad-e44a-4c09-ab15-901bef1dee51 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:33.997767 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:34.033602 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc712b-ffb8-4b4c-b581-d03eb77bb148 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.034239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc712b-ffb8-4b4c-b581-d03eb77bb148 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.034810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc712b-ffb8-4b4c-b581-d03eb77bb148 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.036620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 228/458] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:40:34.049476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be79fa1-78f7-4910-b917-1b96c33c534b None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.049829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be79fa1-78f7-4910-b917-1b96c33c534b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.050040 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be79fa1-78f7-4910-b917-1b96c33c534b None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.051545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 229/459] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:40:34.053201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1fbe1-f587-418b-8616-ab70b191919a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.053767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1fbe1-f587-418b-8616-ab70b191919a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.054244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1fbe1-f587-418b-8616-ab70b191919a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.069375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0167ad-e44a-4c09-ab15-901bef1dee51 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:34.077392 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-dca1fbe1-f587-418b-8616-ab70b191919a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:34.078843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dca1fbe1-f587-418b-8616-ab70b191919a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:34.111697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333dc5a-1a56-4d5d-b706-573eeb5c332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:34.112647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333dc5a-1a56-4d5d-b706-573eeb5c332c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.113033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333dc5a-1a56-4d5d-b706-573eeb5c332c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.113353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333dc5a-1a56-4d5d-b706-573eeb5c332c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.125429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333dc5a-1a56-4d5d-b706-573eeb5c332c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:34.126968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333dc5a-1a56-4d5d-b706-573eeb5c332c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-b333dc5a-1a56-4d5d-b706-573eeb5c332c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:34.147794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:34.148915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:34.149966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:34.149966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:34.149966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:34.149966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:34.160349 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 231/460] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:34.160546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 230/461] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:34.160546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7dd3-608d-44aa-82ca-1ec4cf9450e0 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.160546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7dd3-608d-44aa-82ca-1ec4cf9450e0 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.160546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff7dd3-608d-44aa-82ca-1ec4cf9450e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.161469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333dc5a-1a56-4d5d-b706-573eeb5c332c None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:34.186584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-bfff7dd3-608d-44aa-82ca-1ec4cf9450e0 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:34.188710 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bfff7dd3-608d-44aa-82ca-1ec4cf9450e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:34.221317 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:34.312430 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 231/462] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:34.312430 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff7dd3-608d-44aa-82ca-1ec4cf9450e0 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:34.337038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333dc5a-1a56-4d5d-b706-573eeb5c332c None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:34.338905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:34.376701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0ef38-fde2-426e-aef2-236fb0c9a7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:34.377390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0ef38-fde2-426e-aef2-236fb0c9a7a5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.377700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0ef38-fde2-426e-aef2-236fb0c9a7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.377977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0ef38-fde2-426e-aef2-236fb0c9a7a5 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.378743 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0ef38-fde2-426e-aef2-236fb0c9a7a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:34.379051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0ef38-fde2-426e-aef2-236fb0c9a7a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:34.380927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0ef38-fde2-426e-aef2-236fb0c9a7a5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:34.393941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 232/463] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:34.394931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0ef38-fde2-426e-aef2-236fb0c9a7a5 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:34.434155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff7dd3-608d-44aa-82ca-1ec4cf9450e0 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:34.470217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6032a88-c9d7-4538-ab0b-8f4b973d3759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:34.471021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032a88-c9d7-4538-ab0b-8f4b973d3759 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.471299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032a88-c9d7-4538-ab0b-8f4b973d3759 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.471577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032a88-c9d7-4538-ab0b-8f4b973d3759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.483275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6032a88-c9d7-4538-ab0b-8f4b973d3759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:34.484875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6032a88-c9d7-4538-ab0b-8f4b973d3759 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:34.505887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0ef38-fde2-426e-aef2-236fb0c9a7a5 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-b6032a88-c9d7-4538-ab0b-8f4b973d3759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:34.508004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:34.509063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:34.510117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:34.510117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:34.510117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:34.510117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:34.512126 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 232/464] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:34.513612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6032a88-c9d7-4538-ab0b-8f4b973d3759 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:34.522526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:34.552961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4caf9-ae01-421b-9cc4-506d069ba3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:34.554050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4caf9-ae01-421b-9cc4-506d069ba3e6 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.554474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4caf9-ae01-421b-9cc4-506d069ba3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.554884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4caf9-ae01-421b-9cc4-506d069ba3e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.566221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4caf9-ae01-421b-9cc4-506d069ba3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:34.568123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4caf9-ae01-421b-9cc4-506d069ba3e6 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-9fc4caf9-ae01-421b-9cc4-506d069ba3e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:34.595246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:34.596778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:34.599323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:34.599323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:34.599323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:34.599323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:34.599323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 233/465] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:34.599870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b57f-24a5-4586-b4df-434cc84e7a21 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.600132 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b57f-24a5-4586-b4df-434cc84e7a21 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.600447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b57f-24a5-4586-b4df-434cc84e7a21 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.601908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 234/466] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:34.605629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b57f-24a5-4586-b4df-434cc84e7a21 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:34.630116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:34.643249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6032a88-c9d7-4538-ab0b-8f4b973d3759 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:34.687012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d59a9-b5ca-4b5b-89f8-4dd416ac2e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:34.687706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d59a9-b5ca-4b5b-89f8-4dd416ac2e75 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.687990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d59a9-b5ca-4b5b-89f8-4dd416ac2e75 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.688270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d59a9-b5ca-4b5b-89f8-4dd416ac2e75 None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.689026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d59a9-b5ca-4b5b-89f8-4dd416ac2e75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:34.690928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d59a9-b5ca-4b5b-89f8-4dd416ac2e75 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:34.713665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 233/467] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:40:34 2022] POST /identity/v3/users => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:40:34.715314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d59a9-b5ca-4b5b-89f8-4dd416ac2e75 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:34.748797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b57f-24a5-4586-b4df-434cc84e7a21 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:34.799356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbb27e-21e9-4e1e-8482-f4bf7899795e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:34.800525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb27e-21e9-4e1e-8482-f4bf7899795e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.801154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb27e-21e9-4e1e-8482-f4bf7899795e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.801627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb27e-21e9-4e1e-8482-f4bf7899795e None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.802875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbb27e-21e9-4e1e-8482-f4bf7899795e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:34.803525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbb27e-21e9-4e1e-8482-f4bf7899795e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:34.805521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbb27e-21e9-4e1e-8482-f4bf7899795e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:34.814195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:34.826970 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 235/468] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:34.828958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a40dd3-8bb2-4780-8c23-43434dfd623b None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.829458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a40dd3-8bb2-4780-8c23-43434dfd623b None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.830016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a40dd3-8bb2-4780-8c23-43434dfd623b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.850654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-48a40dd3-8bb2-4780-8c23-43434dfd623b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:34.852723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-48a40dd3-8bb2-4780-8c23-43434dfd623b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:34.878291 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d59a9-b5ca-4b5b-89f8-4dd416ac2e75 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:34.935052 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07527c-3a01-4c8d-92df-648d29a0c29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:34.935387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07527c-3a01-4c8d-92df-648d29a0c29e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:34.937064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07527c-3a01-4c8d-92df-648d29a0c29e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:34.937064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07527c-3a01-4c8d-92df-648d29a0c29e None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:34.937064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07527c-3a01-4c8d-92df-648d29a0c29e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:34.937306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07527c-3a01-4c8d-92df-648d29a0c29e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:34.938558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07527c-3a01-4c8d-92df-648d29a0c29e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:34.940043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 236/469] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:34.940941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a40dd3-8bb2-4780-8c23-43434dfd623b None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:34.945954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:34.967371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 234/470] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:34.968855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07527c-3a01-4c8d-92df-648d29a0c29e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:35.091273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a40dd3-8bb2-4780-8c23-43434dfd623b None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:35.104695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07527c-3a01-4c8d-92df-648d29a0c29e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:35.123827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:35.141853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac484153-8871-4d7c-8551-8e96e719f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:35.142651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac484153-8871-4d7c-8551-8e96e719f765 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:35.142973 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac484153-8871-4d7c-8551-8e96e719f765 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:35.143252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac484153-8871-4d7c-8551-8e96e719f765 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:35.143960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac484153-8871-4d7c-8551-8e96e719f765 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:35.145505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac484153-8871-4d7c-8551-8e96e719f765 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:35.151905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6dade7-9cab-44ce-bd70-a11bcb1c660b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:35.152816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dade7-9cab-44ce-bd70-a11bcb1c660b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:35.153103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dade7-9cab-44ce-bd70-a11bcb1c660b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:35.153383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dade7-9cab-44ce-bd70-a11bcb1c660b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:35.165954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6dade7-9cab-44ce-bd70-a11bcb1c660b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:35.167471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6dade7-9cab-44ce-bd70-a11bcb1c660b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:35.176855 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 237/471] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:40:34 2022] POST /identity/v3/users => generated 248 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:40:35.178684 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac484153-8871-4d7c-8551-8e96e719f765 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-4c6dade7-9cab-44ce-bd70-a11bcb1c660b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:35.188184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:35.189215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:35.190088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:35.190088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:35.190088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:35.190088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:35.190088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 235/472] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:35.191212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d5aa5-1e49-425f-a652-09ce616d21ec None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:35.191502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d5aa5-1e49-425f-a652-09ce616d21ec None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:35.191781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d5aa5-1e49-425f-a652-09ce616d21ec None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:35.193728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 236/473] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:35.199343 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d5aa5-1e49-425f-a652-09ce616d21ec None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:35.269220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:35.335823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac484153-8871-4d7c-8551-8e96e719f765 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:35.342815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d5aa5-1e49-425f-a652-09ce616d21ec None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:35.381536 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffeb8c5-4e1d-4235-b0ef-fe7c8a2f827b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:35.382221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffeb8c5-4e1d-4235-b0ef-fe7c8a2f827b None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:35.382509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffeb8c5-4e1d-4235-b0ef-fe7c8a2f827b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:35.382787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffeb8c5-4e1d-4235-b0ef-fe7c8a2f827b None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:35.383500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffeb8c5-4e1d-4235-b0ef-fe7c8a2f827b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:35.385018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffeb8c5-4e1d-4235-b0ef-fe7c8a2f827b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:35.402597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4be946-fcbe-4c4e-a7c0-bf3336e19cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:35.403284 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4be946-fcbe-4c4e-a7c0-bf3336e19cad None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:35.403563 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4be946-fcbe-4c4e-a7c0-bf3336e19cad None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:35.403840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4be946-fcbe-4c4e-a7c0-bf3336e19cad None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:35.412470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4be946-fcbe-4c4e-a7c0-bf3336e19cad None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:35.412470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4be946-fcbe-4c4e-a7c0-bf3336e19cad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:35.412470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4be946-fcbe-4c4e-a7c0-bf3336e19cad None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:35.419151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 238/474] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:40:35 2022] POST /identity/v3/users => generated 248 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:40:35.420752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be694ad-9d0a-4925-bf3c-f5c7d8ef77ef None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:35.421361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be694ad-9d0a-4925-bf3c-f5c7d8ef77ef None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:35.421790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be694ad-9d0a-4925-bf3c-f5c7d8ef77ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:35.423525 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 237/475] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:35.437716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4be946-fcbe-4c4e-a7c0-bf3336e19cad None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:35.444646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-4be694ad-9d0a-4925-bf3c-f5c7d8ef77ef None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:35.446374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4be694ad-9d0a-4925-bf3c-f5c7d8ef77ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:35.530565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:35.548365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 239/476] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:35.575063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be694ad-9d0a-4925-bf3c-f5c7d8ef77ef None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:35.632831 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4be946-fcbe-4c4e-a7c0-bf3336e19cad None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:35.644849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:35.672768 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789068a-2b49-4892-ae5c-6cb556b3b240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:35.673790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789068a-2b49-4892-ae5c-6cb556b3b240 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:35.674338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789068a-2b49-4892-ae5c-6cb556b3b240 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:35.674713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789068a-2b49-4892-ae5c-6cb556b3b240 None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:35.675600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789068a-2b49-4892-ae5c-6cb556b3b240 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:35.678185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789068a-2b49-4892-ae5c-6cb556b3b240 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:35.720925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be694ad-9d0a-4925-bf3c-f5c7d8ef77ef None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:35.724539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 238/477] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:40:35 2022] POST /identity/v3/users => generated 249 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:40:35.782870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b1fa-97e8-41c3-965d-05e3c20c47a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:35.784135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b1fa-97e8-41c3-965d-05e3c20c47a9 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:35.784720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b1fa-97e8-41c3-965d-05e3c20c47a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:35.785214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b1fa-97e8-41c3-965d-05e3c20c47a9 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:35.787326 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67b1fa-97e8-41c3-965d-05e3c20c47a9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:35.789695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67b1fa-97e8-41c3-965d-05e3c20c47a9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:35.800533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.sql.core [None req-bc67b1fa-97e8-41c3-965d-05e3c20c47a9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 20:40:35.800533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 19 20:40:35.800533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [parameters: {'id': 'ff8336b024144480a4e437f7696c46b9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 19 20:40:35.800533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86668) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-bc67b1fa-97e8-41c3-965d-05e3c20c47a9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result.read() Jul 19 20:40:35.822004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return ref.to_dict() Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:40:35.823011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application yield resource Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise self.value Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application session.commit() Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 20:40:35.824203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.session.flush() Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application compat.raise_( Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 20:40:35.826393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:40:35.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result.read() Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:40:35.829010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [parameters: {'id': 'ff8336b024144480a4e437f7696c46b9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:35.830511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 19 20:40:35.838196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:35.840130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 240/478] 10.222.0.85 () {60 vars in 1131 bytes} [Tue Jul 19 20:40:35 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:35.840266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789068a-2b49-4892-ae5c-6cb556b3b240 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:35.968863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:35.979533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789068a-2b49-4892-ae5c-6cb556b3b240 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:36.023257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907f7f7-ef21-4b2d-9c7b-f80853b94840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:36.024121 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907f7f7-ef21-4b2d-9c7b-f80853b94840 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:36.024465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907f7f7-ef21-4b2d-9c7b-f80853b94840 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:36.024744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907f7f7-ef21-4b2d-9c7b-f80853b94840 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:36.037576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907f7f7-ef21-4b2d-9c7b-f80853b94840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-8907f7f7-ef21-4b2d-9c7b-f80853b94840 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:36.046618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:36.047588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:36.048510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 239/479] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:40:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:36.054045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b1fa-97e8-41c3-965d-05e3c20c47a9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:36.185017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b1fa-97e8-41c3-965d-05e3c20c47a9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:36.198130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:36.227563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bdb44-ac1d-4c2f-9f42-675399301e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:36.228385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bdb44-ac1d-4c2f-9f42-675399301e80 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:36.228844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bdb44-ac1d-4c2f-9f42-675399301e80 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:36.229177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bdb44-ac1d-4c2f-9f42-675399301e80 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:36.230131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bdb44-ac1d-4c2f-9f42-675399301e80 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:36.230446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bdb44-ac1d-4c2f-9f42-675399301e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:36.231891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bdb44-ac1d-4c2f-9f42-675399301e80 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:36.243358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 241/480] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:37.146154 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdeff5-4c08-4360-9a5d-595d4ee97b91 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:37.146676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdeff5-4c08-4360-9a5d-595d4ee97b91 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:37.146749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdeff5-4c08-4360-9a5d-595d4ee97b91 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:37.148486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 240/481] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:37.153718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153ed5c-63c2-4f7a-8819-4e8fe34414d1 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:37.154123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153ed5c-63c2-4f7a-8819-4e8fe34414d1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:37.154292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153ed5c-63c2-4f7a-8819-4e8fe34414d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:37.176612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-7153ed5c-63c2-4f7a-8819-4e8fe34414d1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:37.177820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7153ed5c-63c2-4f7a-8819-4e8fe34414d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:37.274460 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 242/482] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:37.318088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdeff5-4c08-4360-9a5d-595d4ee97b91 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:37.456054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdeff5-4c08-4360-9a5d-595d4ee97b91 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:37.501338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6362c0-74de-487c-a35c-ac56a38fdb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:37.502143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6362c0-74de-487c-a35c-ac56a38fdb12 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:37.502429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6362c0-74de-487c-a35c-ac56a38fdb12 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:37.502709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6362c0-74de-487c-a35c-ac56a38fdb12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:37.521847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6362c0-74de-487c-a35c-ac56a38fdb12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-4c6362c0-74de-487c-a35c-ac56a38fdb12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:37.529689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:37.531148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:37.533196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 241/483] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:37.537490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153ed5c-63c2-4f7a-8819-4e8fe34414d1 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:37.717005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153ed5c-63c2-4f7a-8819-4e8fe34414d1 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:37.720785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:37.773039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f92e5a-27dd-4564-a0b8-d6090848a2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:37.774161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f92e5a-27dd-4564-a0b8-d6090848a2bf None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:37.774685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f92e5a-27dd-4564-a0b8-d6090848a2bf None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:37.775210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f92e5a-27dd-4564-a0b8-d6090848a2bf None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:37.776230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f92e5a-27dd-4564-a0b8-d6090848a2bf None admin] RBAC: Adding query filter params (name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:37.776885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f92e5a-27dd-4564-a0b8-d6090848a2bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:37.778973 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f92e5a-27dd-4564-a0b8-d6090848a2bf None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:37.793151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 243/484] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:37.800305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6362c0-74de-487c-a35c-ac56a38fdb12 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:37.820073 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:37.952849 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6362c0-74de-487c-a35c-ac56a38fdb12 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:37.989998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622c152-2066-4d7c-a0dc-a30eccebe919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:37.990716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622c152-2066-4d7c-a0dc-a30eccebe919 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:37.990998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622c152-2066-4d7c-a0dc-a30eccebe919 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:37.991276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622c152-2066-4d7c-a0dc-a30eccebe919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.003954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622c152-2066-4d7c-a0dc-a30eccebe919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:38.006026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622c152-2066-4d7c-a0dc-a30eccebe919 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-1622c152-2066-4d7c-a0dc-a30eccebe919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:38.024096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:38.025407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:38.026837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:38.026837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:38.026837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:38.026837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:38.026837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 242/485] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:38.031321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f92e5a-27dd-4564-a0b8-d6090848a2bf None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:38.049024 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693cc5e-6f87-4ffe-a026-cedbdb9d5099 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.052358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693cc5e-6f87-4ffe-a026-cedbdb9d5099 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.052358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693cc5e-6f87-4ffe-a026-cedbdb9d5099 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.053344 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 243/486] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:38.059985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dd939-f0ab-43c7-b310-3ad9f6887f40 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.060283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dd939-f0ab-43c7-b310-3ad9f6887f40 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.060567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dd939-f0ab-43c7-b310-3ad9f6887f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.095145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-cf3dd939-f0ab-43c7-b310-3ad9f6887f40 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:38.096375 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cf3dd939-f0ab-43c7-b310-3ad9f6887f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:38.206689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:38.234514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f92e5a-27dd-4564-a0b8-d6090848a2bf None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:38.241619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 244/487] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:38.288229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dd939-f0ab-43c7-b310-3ad9f6887f40 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:38.298338 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52f5f2-544d-4102-b13e-26315c143860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:38.299017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52f5f2-544d-4102-b13e-26315c143860 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.299302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52f5f2-544d-4102-b13e-26315c143860 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.299582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52f5f2-544d-4102-b13e-26315c143860 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.300301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52f5f2-544d-4102-b13e-26315c143860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:38.300643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52f5f2-544d-4102-b13e-26315c143860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:38.302666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52f5f2-544d-4102-b13e-26315c143860 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:38.319703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 244/488] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:38.321118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc591e60-3fdc-4cdb-bf5d-6232dee11999 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.321411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc591e60-3fdc-4cdb-bf5d-6232dee11999 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.321689 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc591e60-3fdc-4cdb-bf5d-6232dee11999 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.324740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 245/489] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:38.325519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc591e60-3fdc-4cdb-bf5d-6232dee11999 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:38.355560 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:38.414937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dd939-f0ab-43c7-b310-3ad9f6887f40 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:38.449581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc591e60-3fdc-4cdb-bf5d-6232dee11999 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:38.464287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66d855-ac72-4d00-b67f-ff9b63cb4d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:38.465216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66d855-ac72-4d00-b67f-ff9b63cb4d5e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.465579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66d855-ac72-4d00-b67f-ff9b63cb4d5e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.465920 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66d855-ac72-4d00-b67f-ff9b63cb4d5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.481420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66d855-ac72-4d00-b67f-ff9b63cb4d5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:38.490605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71aa6f7-4b9c-47c2-a5ed-51fccae14df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-bc66d855-ac72-4d00-b67f-ff9b63cb4d5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:38.491151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:38.492080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:38.492938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 245/490] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:38.494488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecee78-18d3-4bba-a9b7-8521e63c33a3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.494839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecee78-18d3-4bba-a9b7-8521e63c33a3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.495181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecee78-18d3-4bba-a9b7-8521e63c33a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.495873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71aa6f7-4b9c-47c2-a5ed-51fccae14df4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.496687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71aa6f7-4b9c-47c2-a5ed-51fccae14df4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.497219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71aa6f7-4b9c-47c2-a5ed-51fccae14df4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.506798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71aa6f7-4b9c-47c2-a5ed-51fccae14df4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:38.508673 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71aa6f7-4b9c-47c2-a5ed-51fccae14df4 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:38.523222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-45ecee78-18d3-4bba-a9b7-8521e63c33a3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:38.524816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-45ecee78-18d3-4bba-a9b7-8521e63c33a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-c71aa6f7-4b9c-47c2-a5ed-51fccae14df4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:38.537811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:38.539312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:38.540253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:38.540253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:38.540253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:38.540253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:38.540253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:38.540253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:38.540253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:38.540253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:38.540253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:40:38.540253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:38.540253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 246/491] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jul 19 20:40:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:38.541112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71aa6f7-4b9c-47c2-a5ed-51fccae14df4 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:38.597689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:38.637566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 246/492] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:38.639039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ecee78-18d3-4bba-a9b7-8521e63c33a3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:38.660274 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:38.681465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71aa6f7-4b9c-47c2-a5ed-51fccae14df4 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:38.722458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c9e9d-04d5-459c-8530-fe9515d46a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:38.724107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c9e9d-04d5-459c-8530-fe9515d46a19 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.724662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c9e9d-04d5-459c-8530-fe9515d46a19 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.725160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c9e9d-04d5-459c-8530-fe9515d46a19 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.726357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c9e9d-04d5-459c-8530-fe9515d46a19 None admin] RBAC: Adding query filter params (name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:38.726906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c9e9d-04d5-459c-8530-fe9515d46a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:38.728509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c9e9d-04d5-459c-8530-fe9515d46a19 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:38.743713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 247/493] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:38.744544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c9e9d-04d5-459c-8530-fe9515d46a19 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:38.773740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ecee78-18d3-4bba-a9b7-8521e63c33a3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:38.814508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fed08-cd78-49a6-afb3-8d37f799b890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:38.815202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fed08-cd78-49a6-afb3-8d37f799b890 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.815484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fed08-cd78-49a6-afb3-8d37f799b890 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.815749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fed08-cd78-49a6-afb3-8d37f799b890 None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.816640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fed08-cd78-49a6-afb3-8d37f799b890 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:38.816958 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fed08-cd78-49a6-afb3-8d37f799b890 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:38.818994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fed08-cd78-49a6-afb3-8d37f799b890 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:38.865149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 247/494] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jul 19 20:40:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:38.868056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fed08-cd78-49a6-afb3-8d37f799b890 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:38.868227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c9e9d-04d5-459c-8530-fe9515d46a19 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:38.907825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabebeb3-74ee-4b94-9f2e-265b7b8664a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:38.908713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabebeb3-74ee-4b94-9f2e-265b7b8664a0 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.908996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabebeb3-74ee-4b94-9f2e-265b7b8664a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.909265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabebeb3-74ee-4b94-9f2e-265b7b8664a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.913222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:38.922332 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabebeb3-74ee-4b94-9f2e-265b7b8664a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-eabebeb3-74ee-4b94-9f2e-265b7b8664a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:38.930841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:38.932309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 248/495] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:38.939160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc85959-a91b-4111-aeef-de632234ee8c None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:38.939917 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc85959-a91b-4111-aeef-de632234ee8c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:38.939917 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc85959-a91b-4111-aeef-de632234ee8c None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:38.939917 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 249/496] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:38.939917 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc85959-a91b-4111-aeef-de632234ee8c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:38.949777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:39.008609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fed08-cd78-49a6-afb3-8d37f799b890 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:39.055258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558886b-9cbf-4ec4-afbf-50d28322bad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:39.056065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558886b-9cbf-4ec4-afbf-50d28322bad8 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.056378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558886b-9cbf-4ec4-afbf-50d28322bad8 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.057236 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558886b-9cbf-4ec4-afbf-50d28322bad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.070532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558886b-9cbf-4ec4-afbf-50d28322bad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:39.072025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558886b-9cbf-4ec4-afbf-50d28322bad8 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:39.072489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc85959-a91b-4111-aeef-de632234ee8c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-0558886b-9cbf-4ec4-afbf-50d28322bad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:39.094410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:39.095399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:39.096300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:39.096300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:39.096300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:39.096300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:39.096300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 248/497] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:39.096593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558886b-9cbf-4ec4-afbf-50d28322bad8 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:39.115065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b21364-b426-47af-a88a-8b71d36c39c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:39.115712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21364-b426-47af-a88a-8b71d36c39c4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.116003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21364-b426-47af-a88a-8b71d36c39c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.116284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21364-b426-47af-a88a-8b71d36c39c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.185715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b21364-b426-47af-a88a-8b71d36c39c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:39.186628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:39.187184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b21364-b426-47af-a88a-8b71d36c39c4 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:39.195850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:39.199751 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558886b-9cbf-4ec4-afbf-50d28322bad8 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:39.238182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980c4cd-52b6-4cd6-bc09-3120909fc2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:39.238844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980c4cd-52b6-4cd6-bc09-3120909fc2b9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.239715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980c4cd-52b6-4cd6-bc09-3120909fc2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.240184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980c4cd-52b6-4cd6-bc09-3120909fc2b9 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.241304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980c4cd-52b6-4cd6-bc09-3120909fc2b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:39.241612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980c4cd-52b6-4cd6-bc09-3120909fc2b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:39.243021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980c4cd-52b6-4cd6-bc09-3120909fc2b9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:39.251595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 249/498] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:39.252939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851473f-6bf0-45d5-8f6c-cd0119b84804 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.253227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851473f-6bf0-45d5-8f6c-cd0119b84804 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.253502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851473f-6bf0-45d5-8f6c-cd0119b84804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.272703 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-b851473f-6bf0-45d5-8f6c-cd0119b84804 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:39.273852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b851473f-6bf0-45d5-8f6c-cd0119b84804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:39.291835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 250/499] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:39.292747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b21364-b426-47af-a88a-8b71d36c39c4 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:39.346881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 250/500] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:39.348227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bcf27-5ebb-449b-9d81-b2ad4183f280 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.348560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bcf27-5ebb-449b-9d81-b2ad4183f280 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.348843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bcf27-5ebb-449b-9d81-b2ad4183f280 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.354235 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 251/501] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:39.354235 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bcf27-5ebb-449b-9d81-b2ad4183f280 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:39.400569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b21364-b426-47af-a88a-8b71d36c39c4 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:39.408016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:39.436680 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e3445-9d2c-482b-97ef-780919580965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:39.437350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3445-9d2c-482b-97ef-780919580965 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.437631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3445-9d2c-482b-97ef-780919580965 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.437898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e3445-9d2c-482b-97ef-780919580965 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.438608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e3445-9d2c-482b-97ef-780919580965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:39.438909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e3445-9d2c-482b-97ef-780919580965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:39.440258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e3445-9d2c-482b-97ef-780919580965 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:39.441606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:39.452864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 251/502] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:39.453821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e3445-9d2c-482b-97ef-780919580965 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:39.463394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bcf27-5ebb-449b-9d81-b2ad4183f280 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:39.496963 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8ee74-0991-489a-9f10-7b6eb4e8b0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:39.497749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8ee74-0991-489a-9f10-7b6eb4e8b0ab None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.498031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8ee74-0991-489a-9f10-7b6eb4e8b0ab None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.498320 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8ee74-0991-489a-9f10-7b6eb4e8b0ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.509129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8ee74-0991-489a-9f10-7b6eb4e8b0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:39.510576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8ee74-0991-489a-9f10-7b6eb4e8b0ab None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-f2e8ee74-0991-489a-9f10-7b6eb4e8b0ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:39.528736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:39.529717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:39.530654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:39.530654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:39.530654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:39.530654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:39.530654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 252/503] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:39.531294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8f090-c631-4a85-8876-9149a2fc54d7 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.531294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8f090-c631-4a85-8876-9149a2fc54d7 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.531543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8f090-c631-4a85-8876-9149a2fc54d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.550218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-dba8f090-c631-4a85-8876-9149a2fc54d7 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:39.551455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dba8f090-c631-4a85-8876-9149a2fc54d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:39.560803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e3445-9d2c-482b-97ef-780919580965 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:39.593720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d376d21-d500-4eb7-8c8d-5491be3fb4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:39.594515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d376d21-d500-4eb7-8c8d-5491be3fb4ab None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.594799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d376d21-d500-4eb7-8c8d-5491be3fb4ab None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.595075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d376d21-d500-4eb7-8c8d-5491be3fb4ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.606260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d376d21-d500-4eb7-8c8d-5491be3fb4ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:39.607718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d376d21-d500-4eb7-8c8d-5491be3fb4ab None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:39.620743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 253/504] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:39.621599 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8f090-c631-4a85-8876-9149a2fc54d7 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-3d376d21-d500-4eb7-8c8d-5491be3fb4ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:39.626765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:39.627735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:39.628735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:39.628735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:39.628735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:39.628735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:39.628735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 252/505] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:39.629317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d376d21-d500-4eb7-8c8d-5491be3fb4ab None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:39.654512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:39.692411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:39.733409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8f090-c631-4a85-8876-9149a2fc54d7 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:39.752909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d376d21-d500-4eb7-8c8d-5491be3fb4ab None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:39.772195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5971405-4c53-416c-8523-1ba5928ed7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:39.773027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5971405-4c53-416c-8523-1ba5928ed7e4 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.773307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5971405-4c53-416c-8523-1ba5928ed7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.773578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5971405-4c53-416c-8523-1ba5928ed7e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.783793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5971405-4c53-416c-8523-1ba5928ed7e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:39.788628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b295a69-2c63-4800-a3db-fb05b07a5129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:39.789427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b295a69-2c63-4800-a3db-fb05b07a5129 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.789715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b295a69-2c63-4800-a3db-fb05b07a5129 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.789995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b295a69-2c63-4800-a3db-fb05b07a5129 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-a5971405-4c53-416c-8523-1ba5928ed7e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:39.791886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:39.792803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:39.793596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 254/506] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:39.794260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5971405-4c53-416c-8523-1ba5928ed7e4 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:39.799490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b295a69-2c63-4800-a3db-fb05b07a5129 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:39.800990 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b295a69-2c63-4800-a3db-fb05b07a5129 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-3b295a69-2c63-4800-a3db-fb05b07a5129 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:39.816319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:39.817588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:39.818725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:39.818725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:39.818725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:39.818725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:39.818725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:39.818725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:39.818725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:39.818725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:39.818725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:40:39.818725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:39.818725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 253/507] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:39.821191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b295a69-2c63-4800-a3db-fb05b07a5129 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:39.907865 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:39.912026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5971405-4c53-416c-8523-1ba5928ed7e4 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:39.930666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b295a69-2c63-4800-a3db-fb05b07a5129 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:39.936591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:39.948758 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f55b7-bd80-497a-b26a-9daa1515444b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:39.949434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f55b7-bd80-497a-b26a-9daa1515444b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.949714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f55b7-bd80-497a-b26a-9daa1515444b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.949989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f55b7-bd80-497a-b26a-9daa1515444b None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.950718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f55b7-bd80-497a-b26a-9daa1515444b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:39.951023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f55b7-bd80-497a-b26a-9daa1515444b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:39.952451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f55b7-bd80-497a-b26a-9daa1515444b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:39.964953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 255/508] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:39.965787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f55b7-bd80-497a-b26a-9daa1515444b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:39.969627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4128e-fb35-4bec-96e4-e121b7a51e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:39.970168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4128e-fb35-4bec-96e4-e121b7a51e97 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:39.970707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4128e-fb35-4bec-96e4-e121b7a51e97 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:39.970990 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4128e-fb35-4bec-96e4-e121b7a51e97 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:39.971795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4128e-fb35-4bec-96e4-e121b7a51e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:39.972033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4128e-fb35-4bec-96e4-e121b7a51e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:39.973553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4128e-fb35-4bec-96e4-e121b7a51e97 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:39.988737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 254/509] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:39.989683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4128e-fb35-4bec-96e4-e121b7a51e97 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:40.086348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f55b7-bd80-497a-b26a-9daa1515444b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:40.093435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4128e-fb35-4bec-96e4-e121b7a51e97 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:40.120730 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167b0fb-6ba3-4b1e-94e7-1bd01bb4d7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:40.121384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167b0fb-6ba3-4b1e-94e7-1bd01bb4d7d1 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:40.121664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167b0fb-6ba3-4b1e-94e7-1bd01bb4d7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:40.121946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167b0fb-6ba3-4b1e-94e7-1bd01bb4d7d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:40.128828 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd291c-26ff-4cf6-98aa-b6303a2902ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:40.129491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd291c-26ff-4cf6-98aa-b6303a2902ad None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:40.129810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd291c-26ff-4cf6-98aa-b6303a2902ad None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:40.130098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd291c-26ff-4cf6-98aa-b6303a2902ad None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:40.130871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd291c-26ff-4cf6-98aa-b6303a2902ad None admin] RBAC: Adding query filter params (name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:40.131180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd291c-26ff-4cf6-98aa-b6303a2902ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:40.132230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167b0fb-6ba3-4b1e-94e7-1bd01bb4d7d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:40.132580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd291c-26ff-4cf6-98aa-b6303a2902ad None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:40.133684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167b0fb-6ba3-4b1e-94e7-1bd01bb4d7d1 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:40.137861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:40.141576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 255/510] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:40.142727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd291c-26ff-4cf6-98aa-b6303a2902ad None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-9167b0fb-6ba3-4b1e-94e7-1bd01bb4d7d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:40.156425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:40.157481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:40.158540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:40.158540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:40.158540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:40.158540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:40.158540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 256/511] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:40.158540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167b0fb-6ba3-4b1e-94e7-1bd01bb4d7d1 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:40.197771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:40.262293 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd291c-26ff-4cf6-98aa-b6303a2902ad None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:40.290000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167b0fb-6ba3-4b1e-94e7-1bd01bb4d7d1 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:40.301960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dfb3b-47b5-4d07-b9e4-1626eeecd890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:40.302625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dfb3b-47b5-4d07-b9e4-1626eeecd890 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:40.302905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dfb3b-47b5-4d07-b9e4-1626eeecd890 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:40.303190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dfb3b-47b5-4d07-b9e4-1626eeecd890 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:40.304014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dfb3b-47b5-4d07-b9e4-1626eeecd890 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:40.304342 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dfb3b-47b5-4d07-b9e4-1626eeecd890 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:40.305768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dfb3b-47b5-4d07-b9e4-1626eeecd890 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:40.326496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8078bf-229f-409e-b4dd-6923bf7db4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:40.327289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8078bf-229f-409e-b4dd-6923bf7db4a0 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:40.327577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8078bf-229f-409e-b4dd-6923bf7db4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:40.327859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8078bf-229f-409e-b4dd-6923bf7db4a0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:40.336231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8078bf-229f-409e-b4dd-6923bf7db4a0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:40.337769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8078bf-229f-409e-b4dd-6923bf7db4a0 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:40.343696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 256/512] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jul 19 20:40:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:40.344556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dfb3b-47b5-4d07-b9e4-1626eeecd890 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-9f8078bf-229f-409e-b4dd-6923bf7db4a0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:40.355349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:40.356381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 257/513] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:40.357728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8078bf-229f-409e-b4dd-6923bf7db4a0 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:40.415093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:40.457665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:40.459323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dfb3b-47b5-4d07-b9e4-1626eeecd890 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:40.476633 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8078bf-229f-409e-b4dd-6923bf7db4a0 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:40.494783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12fc53-8d6f-449c-b9f4-603b4c1303b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:40.495584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12fc53-8d6f-449c-b9f4-603b4c1303b3 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:40.495864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12fc53-8d6f-449c-b9f4-603b4c1303b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:40.496138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12fc53-8d6f-449c-b9f4-603b4c1303b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:40.502923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12fc53-8d6f-449c-b9f4-603b4c1303b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:40.504403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12fc53-8d6f-449c-b9f4-603b4c1303b3 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:40.511639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c14f3-9aef-49d2-94aa-cb84b085b75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:40.512465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c14f3-9aef-49d2-94aa-cb84b085b75f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:40.512753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c14f3-9aef-49d2-94aa-cb84b085b75f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:40.513030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c14f3-9aef-49d2-94aa-cb84b085b75f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-9c12fc53-8d6f-449c-b9f4-603b4c1303b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:40.519216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:40.520090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:40.521563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:40.521563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:40.521563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:40.521563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:40.521563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:40.521563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:40.521563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:40.521563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:40.521563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 257/514] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:40.522048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12fc53-8d6f-449c-b9f4-603b4c1303b3 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:40.526505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c14f3-9aef-49d2-94aa-cb84b085b75f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:40.527992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c14f3-9aef-49d2-94aa-cb84b085b75f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-215c14f3-9aef-49d2-94aa-cb84b085b75f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:40.546358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:40.547436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:40.548503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:40.548503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:40.548503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:40.548503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:40.548503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 258/515] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:40.548823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c14f3-9aef-49d2-94aa-cb84b085b75f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:40.653532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12fc53-8d6f-449c-b9f4-603b4c1303b3 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:40.660147 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c14f3-9aef-49d2-94aa-cb84b085b75f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:40.671506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:40.691569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491af3da-d2c2-4f00-aeb0-35903c579439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:40.692972 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491af3da-d2c2-4f00-aeb0-35903c579439 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:40.693260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491af3da-d2c2-4f00-aeb0-35903c579439 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:40.693533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491af3da-d2c2-4f00-aeb0-35903c579439 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:40.694250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491af3da-d2c2-4f00-aeb0-35903c579439 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:40.694553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491af3da-d2c2-4f00-aeb0-35903c579439 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:40.695932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491af3da-d2c2-4f00-aeb0-35903c579439 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:40.702871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaede09-1a10-425c-b929-fbc0bb6db710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:40.703513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaede09-1a10-425c-b929-fbc0bb6db710 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:40.703792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaede09-1a10-425c-b929-fbc0bb6db710 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:40.704067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaede09-1a10-425c-b929-fbc0bb6db710 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:40.708599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 258/516] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:40.709477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491af3da-d2c2-4f00-aeb0-35903c579439 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:40.726290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:40.789072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaede09-1a10-425c-b929-fbc0bb6db710 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:40.790559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaede09-1a10-425c-b929-fbc0bb6db710 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:40.818481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491af3da-d2c2-4f00-aeb0-35903c579439 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:40.853510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25d940-1e81-4a5c-87f4-48a06e7868ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:40.854178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25d940-1e81-4a5c-87f4-48a06e7868ef None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:40.854467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25d940-1e81-4a5c-87f4-48a06e7868ef None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:40.854741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25d940-1e81-4a5c-87f4-48a06e7868ef None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:40.855592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25d940-1e81-4a5c-87f4-48a06e7868ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:40.855903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25d940-1e81-4a5c-87f4-48a06e7868ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:40.857376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25d940-1e81-4a5c-87f4-48a06e7868ef None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:40.875178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 259/517] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jul 19 20:40:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:40.876019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25d940-1e81-4a5c-87f4-48a06e7868ef None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:40.897793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 259/518] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:40.898647 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaede09-1a10-425c-b929-fbc0bb6db710 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:40.918944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:40.950598 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:40.986213 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25d940-1e81-4a5c-87f4-48a06e7868ef None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:41.022130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaede09-1a10-425c-b929-fbc0bb6db710 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:41.022329 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06ec26-dd41-4c01-bd15-725f4cc1d8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:41.022944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06ec26-dd41-4c01-bd15-725f4cc1d8b7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:41.023227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06ec26-dd41-4c01-bd15-725f4cc1d8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:41.023504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06ec26-dd41-4c01-bd15-725f4cc1d8b7 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:41.024286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06ec26-dd41-4c01-bd15-725f4cc1d8b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:41.024772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06ec26-dd41-4c01-bd15-725f4cc1d8b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:41.026410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06ec26-dd41-4c01-bd15-725f4cc1d8b7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:41.045322 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 260/519] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:41.046130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06ec26-dd41-4c01-bd15-725f4cc1d8b7 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:41.059827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec04922-93c3-48ad-8df0-7166d65b31fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:41.060540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04922-93c3-48ad-8df0-7166d65b31fc None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:41.060820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04922-93c3-48ad-8df0-7166d65b31fc None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:41.061089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec04922-93c3-48ad-8df0-7166d65b31fc None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:41.061807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec04922-93c3-48ad-8df0-7166d65b31fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:41.062103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec04922-93c3-48ad-8df0-7166d65b31fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:41.063501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec04922-93c3-48ad-8df0-7166d65b31fc None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:41.075895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 260/520] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:41.076972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec04922-93c3-48ad-8df0-7166d65b31fc None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:41.151675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:41.159350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06ec26-dd41-4c01-bd15-725f4cc1d8b7 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:41.195854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d10d26-4cb5-46d5-a012-a24ed295b111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:41.196412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec04922-93c3-48ad-8df0-7166d65b31fc None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:41.196529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d10d26-4cb5-46d5-a012-a24ed295b111 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:41.196753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d10d26-4cb5-46d5-a012-a24ed295b111 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:41.197031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d10d26-4cb5-46d5-a012-a24ed295b111 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:41.197752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d10d26-4cb5-46d5-a012-a24ed295b111 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:41.199220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d10d26-4cb5-46d5-a012-a24ed295b111 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:41.203836 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:41.223100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 261/521] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:40:41 2022] POST /identity/v3/users => generated 247 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:40:41.224000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d10d26-4cb5-46d5-a012-a24ed295b111 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:41.236342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e08813-b3b2-4118-ade8-e7988a758c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:41.236698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e08813-b3b2-4118-ade8-e7988a758c36 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:41.237027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e08813-b3b2-4118-ade8-e7988a758c36 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:41.237401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e08813-b3b2-4118-ade8-e7988a758c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:41.325259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e08813-b3b2-4118-ade8-e7988a758c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:41.326980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e08813-b3b2-4118-ade8-e7988a758c36 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:41.353845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d10d26-4cb5-46d5-a012-a24ed295b111 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:41.401427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9688c-9de8-4192-935e-9cec1e744e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:41.402375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9688c-9de8-4192-935e-9cec1e744e32 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:41.402540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9688c-9de8-4192-935e-9cec1e744e32 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:41.402806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9688c-9de8-4192-935e-9cec1e744e32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:41.417042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9688c-9de8-4192-935e-9cec1e744e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:41.418568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9688c-9de8-4192-935e-9cec1e744e32 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-67c9688c-9de8-4192-935e-9cec1e744e32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:41.441162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:41.442129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:41.444252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:41.444252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:41.444252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:41.444252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:41.444252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 262/522] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:41.445085 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9688c-9de8-4192-935e-9cec1e744e32 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:41.449870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:41.464282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 261/523] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:41.465231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e08813-b3b2-4118-ade8-e7988a758c36 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:41.476500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:41.568899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9688c-9de8-4192-935e-9cec1e744e32 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:41.584582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e08813-b3b2-4118-ade8-e7988a758c36 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:41.603880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e6f3c-8da3-43f7-a2db-21141f50d1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:41.604833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e6f3c-8da3-43f7-a2db-21141f50d1a7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:41.605156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e6f3c-8da3-43f7-a2db-21141f50d1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:41.605495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e6f3c-8da3-43f7-a2db-21141f50d1a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:41.606370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e6f3c-8da3-43f7-a2db-21141f50d1a7 None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=928b8b2b510b4c939b37b2c0dfa57060) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:41.606689 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e6f3c-8da3-43f7-a2db-21141f50d1a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:41.608148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e6f3c-8da3-43f7-a2db-21141f50d1a7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:41.615144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 263/524] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:41 2022] GET /identity/v3/role_assignments?user.id=928b8b2b510b4c939b37b2c0dfa57060&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:41.615772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e6f3c-8da3-43f7-a2db-21141f50d1a7 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:41.633658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2684ca-9e5f-4f1c-8303-42cdd4b9b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:41.635199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2684ca-9e5f-4f1c-8303-42cdd4b9b2a6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:41.635695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2684ca-9e5f-4f1c-8303-42cdd4b9b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:41.636952 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2684ca-9e5f-4f1c-8303-42cdd4b9b2a6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:41.648502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2684ca-9e5f-4f1c-8303-42cdd4b9b2a6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:41.651232 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2684ca-9e5f-4f1c-8303-42cdd4b9b2a6 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-dd2684ca-9e5f-4f1c-8303-42cdd4b9b2a6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:41.673189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:41.674186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:41.675077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:41.675077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:41.675077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:41.675077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:41.675077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:41.675077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:41.675077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:41.675077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:41.675077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:40:41.675077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:41.683676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 262/525] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:41.683676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2684ca-9e5f-4f1c-8303-42cdd4b9b2a6 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:41.714291 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:41.767531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e6f3c-8da3-43f7-a2db-21141f50d1a7 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:41.771740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:41.814811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2684ca-9e5f-4f1c-8303-42cdd4b9b2a6 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:41.816360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977b24f-61a5-4d78-992b-b2543628eea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:41.817985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977b24f-61a5-4d78-992b-b2543628eea9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:41.819372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977b24f-61a5-4d78-992b-b2543628eea9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:41.819779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977b24f-61a5-4d78-992b-b2543628eea9 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:41.820724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977b24f-61a5-4d78-992b-b2543628eea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:41.821132 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977b24f-61a5-4d78-992b-b2543628eea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:41.822639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977b24f-61a5-4d78-992b-b2543628eea9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:41.845885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 264/526] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:41.845885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977b24f-61a5-4d78-992b-b2543628eea9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:41.860979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f2790-fd32-40ab-9765-ff991d83671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:41.862039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f2790-fd32-40ab-9765-ff991d83671b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:41.862465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f2790-fd32-40ab-9765-ff991d83671b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:41.862907 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f2790-fd32-40ab-9765-ff991d83671b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:41.876281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f2790-fd32-40ab-9765-ff991d83671b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:41.878052 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f2790-fd32-40ab-9765-ff991d83671b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-c90f2790-fd32-40ab-9765-ff991d83671b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:41.898540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:41.899479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:41.900446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:41.900446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:41.900446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:41.900446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:41.907281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 263/527] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:41.908415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f2790-fd32-40ab-9765-ff991d83671b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:41.980118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977b24f-61a5-4d78-992b-b2543628eea9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:42.017616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:42.019619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f2790-fd32-40ab-9765-ff991d83671b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:42.029265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20debf32-4658-4b5c-9a44-ac3137ba60af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:42.029939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debf32-4658-4b5c-9a44-ac3137ba60af None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:42.030221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debf32-4658-4b5c-9a44-ac3137ba60af None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:42.030492 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debf32-4658-4b5c-9a44-ac3137ba60af None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:42.031804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20debf32-4658-4b5c-9a44-ac3137ba60af None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:42.032113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20debf32-4658-4b5c-9a44-ac3137ba60af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:42.034259 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20debf32-4658-4b5c-9a44-ac3137ba60af None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:42.041519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:42.061327 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b2b43-e3d5-45a8-aa83-859b807db79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:42.062500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b2b43-e3d5-45a8-aa83-859b807db79b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:42.062996 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b2b43-e3d5-45a8-aa83-859b807db79b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:42.063521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b2b43-e3d5-45a8-aa83-859b807db79b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:42.064400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 265/528] 10.222.0.85 () {56 vars in 1139 bytes} [Tue Jul 19 20:40:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:42.065505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20debf32-4658-4b5c-9a44-ac3137ba60af None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:42.075197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b2b43-e3d5-45a8-aa83-859b807db79b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:42.077017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b2b43-e3d5-45a8-aa83-859b807db79b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-724b2b43-e3d5-45a8-aa83-859b807db79b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:42.094001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:42.095035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:42.095905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:42.095905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:42.095905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:42.095905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:42.095905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:42.095905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:42.095905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:42.095905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:42.098179 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 264/529] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:42.099338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b2b43-e3d5-45a8-aa83-859b807db79b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:42.195205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20debf32-4658-4b5c-9a44-ac3137ba60af None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:42.232607 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b2b43-e3d5-45a8-aa83-859b807db79b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:42.242810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98599a2-43ff-4879-aba2-4e7b7ccdb79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:42.243520 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98599a2-43ff-4879-aba2-4e7b7ccdb79f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:42.243849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98599a2-43ff-4879-aba2-4e7b7ccdb79f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:42.244152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98599a2-43ff-4879-aba2-4e7b7ccdb79f None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:42.245427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98599a2-43ff-4879-aba2-4e7b7ccdb79f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:42.245819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98599a2-43ff-4879-aba2-4e7b7ccdb79f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:42.247698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98599a2-43ff-4879-aba2-4e7b7ccdb79f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:42.271429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 266/530] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:42.271429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98599a2-43ff-4879-aba2-4e7b7ccdb79f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:42.280971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc717d5-03e4-4547-959a-257ac806a42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:42.282054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc717d5-03e4-4547-959a-257ac806a42b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:42.282558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc717d5-03e4-4547-959a-257ac806a42b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:42.283030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc717d5-03e4-4547-959a-257ac806a42b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:42.329165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:42.363050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:42.393375 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc717d5-03e4-4547-959a-257ac806a42b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:42.395147 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc717d5-03e4-4547-959a-257ac806a42b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:42.422670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98599a2-43ff-4879-aba2-4e7b7ccdb79f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:42.470333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3de8b-0bdc-4d12-9bf2-5929994c18c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:42.471390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3de8b-0bdc-4d12-9bf2-5929994c18c5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:42.471876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3de8b-0bdc-4d12-9bf2-5929994c18c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:42.472402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3de8b-0bdc-4d12-9bf2-5929994c18c5 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:42.473405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3de8b-0bdc-4d12-9bf2-5929994c18c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:42.473916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3de8b-0bdc-4d12-9bf2-5929994c18c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:42.475663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3de8b-0bdc-4d12-9bf2-5929994c18c5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:42.493197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 267/531] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:42.494095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3de8b-0bdc-4d12-9bf2-5929994c18c5 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:42.524887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 265/532] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:42.526112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc717d5-03e4-4547-959a-257ac806a42b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:42.626010 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:42.639858 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3de8b-0bdc-4d12-9bf2-5929994c18c5 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:42.650524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:42.664671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc717d5-03e4-4547-959a-257ac806a42b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:42.689425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb341c22-bfd4-4905-8593-7e1af1346834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:42.690240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341c22-bfd4-4905-8593-7e1af1346834 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:42.690511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341c22-bfd4-4905-8593-7e1af1346834 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:42.690790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb341c22-bfd4-4905-8593-7e1af1346834 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:42.699869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb341c22-bfd4-4905-8593-7e1af1346834 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:42.701349 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb341c22-bfd4-4905-8593-7e1af1346834 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:42.719961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2c577-027e-43b3-96ac-dc4dcff144bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:42.721028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2c577-027e-43b3-96ac-dc4dcff144bc None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:42.721529 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2c577-027e-43b3-96ac-dc4dcff144bc None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:42.722063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2c577-027e-43b3-96ac-dc4dcff144bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-fb341c22-bfd4-4905-8593-7e1af1346834 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:42.722165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:42.722965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:42.729534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:42.729534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:42.729534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:42.729534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:42.729534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:42.729534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:42.729534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:42.729534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:42.729534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 268/533] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:42.729534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb341c22-bfd4-4905-8593-7e1af1346834 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:42.731372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2c577-027e-43b3-96ac-dc4dcff144bc None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=e35e7720e3bf4c9e80bdefa6a1cb8296) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:42.731892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2c577-027e-43b3-96ac-dc4dcff144bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:42.733903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2c577-027e-43b3-96ac-dc4dcff144bc None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:42.744528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 266/534] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:42 2022] GET /identity/v3/role_assignments?user.id=e35e7720e3bf4c9e80bdefa6a1cb8296&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:42.745764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2c577-027e-43b3-96ac-dc4dcff144bc None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:42.882946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb341c22-bfd4-4905-8593-7e1af1346834 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:42.898783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2c577-027e-43b3-96ac-dc4dcff144bc None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:42.938191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522670ec-7e07-43c8-8b7f-77547b314b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:42.939438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522670ec-7e07-43c8-8b7f-77547b314b7f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:42.940044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522670ec-7e07-43c8-8b7f-77547b314b7f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:42.940547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522670ec-7e07-43c8-8b7f-77547b314b7f None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:42.941605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522670ec-7e07-43c8-8b7f-77547b314b7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:42.942232 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522670ec-7e07-43c8-8b7f-77547b314b7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:42.944109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522670ec-7e07-43c8-8b7f-77547b314b7f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:42.953086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af87a64-eee2-4de3-9367-e9e12fa5a852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:42.953910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af87a64-eee2-4de3-9367-e9e12fa5a852 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:42.954192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af87a64-eee2-4de3-9367-e9e12fa5a852 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:42.954464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af87a64-eee2-4de3-9367-e9e12fa5a852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:42.965723 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 267/535] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:42.972373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af87a64-eee2-4de3-9367-e9e12fa5a852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:42.975499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af87a64-eee2-4de3-9367-e9e12fa5a852 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:42.977278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522670ec-7e07-43c8-8b7f-77547b314b7f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:42.983028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-4af87a64-eee2-4de3-9367-e9e12fa5a852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:42.994267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:42.995381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:42.996489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:42.996489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:42.996489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:42.996489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:43.003964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 269/536] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:43.006506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af87a64-eee2-4de3-9367-e9e12fa5a852 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:43.032070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:43.096281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522670ec-7e07-43c8-8b7f-77547b314b7f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:43.142488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a5051-b449-481d-aa8b-7c7db88178b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:43.143692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a5051-b449-481d-aa8b-7c7db88178b3 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:43.144172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a5051-b449-481d-aa8b-7c7db88178b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:43.144773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a5051-b449-481d-aa8b-7c7db88178b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:43.146349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a5051-b449-481d-aa8b-7c7db88178b3 None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=81411204bc9f4210880d807b41da9643) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:43.146874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a5051-b449-481d-aa8b-7c7db88178b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:43.150420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a5051-b449-481d-aa8b-7c7db88178b3 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:43.162308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 268/537] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:42 2022] GET /identity/v3/role_assignments?user.id=81411204bc9f4210880d807b41da9643&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:43.197338 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af87a64-eee2-4de3-9367-e9e12fa5a852 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:43.264433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbc0a4-12b6-44f8-9d3b-3fdbbc240078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:43.265049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbc0a4-12b6-44f8-9d3b-3fdbbc240078 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:43.265403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbc0a4-12b6-44f8-9d3b-3fdbbc240078 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:43.265764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbc0a4-12b6-44f8-9d3b-3fdbbc240078 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:43.266824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbc0a4-12b6-44f8-9d3b-3fdbbc240078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:43.267174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbc0a4-12b6-44f8-9d3b-3fdbbc240078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:43.268678 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbc0a4-12b6-44f8-9d3b-3fdbbc240078 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:43.296556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 270/538] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:43.303582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a5051-b449-481d-aa8b-7c7db88178b3 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:43.444540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:43.518371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a5051-b449-481d-aa8b-7c7db88178b3 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:43.571136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a0a21-c78e-4bd2-adc2-ea5fadba80cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:43.571955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a0a21-c78e-4bd2-adc2-ea5fadba80cc None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:43.572241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a0a21-c78e-4bd2-adc2-ea5fadba80cc None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:43.572556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a0a21-c78e-4bd2-adc2-ea5fadba80cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:43.582705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a0a21-c78e-4bd2-adc2-ea5fadba80cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:43.585567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a0a21-c78e-4bd2-adc2-ea5fadba80cc None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-e75a0a21-c78e-4bd2-adc2-ea5fadba80cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:43.609558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:43.610806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:43.611656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:43.611656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:43.611656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:43.611656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:43.611656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:43.611656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:43.611656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:43.611656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:43.611656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 269/539] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:43.620821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbc0a4-12b6-44f8-9d3b-3fdbbc240078 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:43.766834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:43.773872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e749c7-2004-4ae1-9be5-084d723732cc None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:43.774151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e749c7-2004-4ae1-9be5-084d723732cc None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:43.774448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e749c7-2004-4ae1-9be5-084d723732cc None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:43.777032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 270/540] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:43.786881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51617a-72c1-48b0-8799-77621b607e36 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:43.787283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51617a-72c1-48b0-8799-77621b607e36 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:43.787590 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51617a-72c1-48b0-8799-77621b607e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:43.802863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbc0a4-12b6-44f8-9d3b-3fdbbc240078 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:43.821759 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3d51617a-72c1-48b0-8799-77621b607e36 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:43.823463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3d51617a-72c1-48b0-8799-77621b607e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:43.868502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4226db1-604f-4674-9d1f-d7560f44524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:43.869217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4226db1-604f-4674-9d1f-d7560f44524e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:43.869500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4226db1-604f-4674-9d1f-d7560f44524e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:43.869778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4226db1-604f-4674-9d1f-d7560f44524e None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:43.870591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4226db1-604f-4674-9d1f-d7560f44524e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:43.871000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4226db1-604f-4674-9d1f-d7560f44524e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:43.873809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4226db1-604f-4674-9d1f-d7560f44524e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:43.905118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 271/541] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:43.911964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4226db1-604f-4674-9d1f-d7560f44524e None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:44.010915 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 271/542] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:44.042528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4226db1-604f-4674-9d1f-d7560f44524e None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:44.062411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51617a-72c1-48b0-8799-77621b607e36 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:44.105453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a9873-9d55-48c5-9b77-d730365eb67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:44.106343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a9873-9d55-48c5-9b77-d730365eb67c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:44.106684 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a9873-9d55-48c5-9b77-d730365eb67c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:44.107012 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a9873-9d55-48c5-9b77-d730365eb67c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:44.108356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a9873-9d55-48c5-9b77-d730365eb67c None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=5db827b3a66843749e04376559aa27c2) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:44.108710 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a9873-9d55-48c5-9b77-d730365eb67c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:44.110578 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a9873-9d55-48c5-9b77-d730365eb67c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:44.118282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 272/543] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:43 2022] GET /identity/v3/role_assignments?user.id=5db827b3a66843749e04376559aa27c2&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:44.120154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e0ca8-0a2a-4592-99db-b88a81b2be72 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:44.120524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e0ca8-0a2a-4592-99db-b88a81b2be72 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:44.120804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e0ca8-0a2a-4592-99db-b88a81b2be72 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:44.122890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 273/544] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:44.129466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0def7-de93-4264-8bc8-3b5621fbfb7e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:44.130584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0def7-de93-4264-8bc8-3b5621fbfb7e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:44.130977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0def7-de93-4264-8bc8-3b5621fbfb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:44.137445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:44.155659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-d8f0def7-de93-4264-8bc8-3b5621fbfb7e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:44.157383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d8f0def7-de93-4264-8bc8-3b5621fbfb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:44.173463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:44.260269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51617a-72c1-48b0-8799-77621b607e36 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:44.319071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a162595-af7e-47d8-b277-3b14f1da1c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:44.320934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a162595-af7e-47d8-b277-3b14f1da1c89 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:44.321626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a162595-af7e-47d8-b277-3b14f1da1c89 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:44.322306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a162595-af7e-47d8-b277-3b14f1da1c89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:44.334003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 274/545] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:40:44.338871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a162595-af7e-47d8-b277-3b14f1da1c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-7a162595-af7e-47d8-b277-3b14f1da1c89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:44.347819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:44.348962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:44.349960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:44.352722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 272/546] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:44.360369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0def7-de93-4264-8bc8-3b5621fbfb7e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:44.390486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a162595-af7e-47d8-b277-3b14f1da1c89 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:44.598206 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0def7-de93-4264-8bc8-3b5621fbfb7e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:44.639254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:44.646653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a162595-af7e-47d8-b277-3b14f1da1c89 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:44.653864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32141151-045a-4046-8616-1c98f94850d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:44.654598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141151-045a-4046-8616-1c98f94850d3 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:44.654892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141151-045a-4046-8616-1c98f94850d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:44.655200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32141151-045a-4046-8616-1c98f94850d3 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:44.655984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32141151-045a-4046-8616-1c98f94850d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:44.656355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32141151-045a-4046-8616-1c98f94850d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:44.657838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32141151-045a-4046-8616-1c98f94850d3 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:44.672766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:44.684362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 275/547] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:44.694587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32141151-045a-4046-8616-1c98f94850d3 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:44.716473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289b6ac-d0a6-4bbe-8cda-1a94f9d92f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:44.717411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289b6ac-d0a6-4bbe-8cda-1a94f9d92f38 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:44.717701 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289b6ac-d0a6-4bbe-8cda-1a94f9d92f38 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:44.718046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289b6ac-d0a6-4bbe-8cda-1a94f9d92f38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:44.741827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289b6ac-d0a6-4bbe-8cda-1a94f9d92f38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-3289b6ac-d0a6-4bbe-8cda-1a94f9d92f38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:44.750795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:44.752049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:44.752912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:44.756505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 273/548] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:44.760023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289b6ac-d0a6-4bbe-8cda-1a94f9d92f38 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:44.919180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289b6ac-d0a6-4bbe-8cda-1a94f9d92f38 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:44.944959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32141151-045a-4046-8616-1c98f94850d3 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:44.956617 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ed3f4-99fc-451f-b9fa-75215020269c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:44.957696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ed3f4-99fc-451f-b9fa-75215020269c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:44.958664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ed3f4-99fc-451f-b9fa-75215020269c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:44.959177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ed3f4-99fc-451f-b9fa-75215020269c None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:44.960245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ed3f4-99fc-451f-b9fa-75215020269c None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:44.960779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ed3f4-99fc-451f-b9fa-75215020269c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:44.962368 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ed3f4-99fc-451f-b9fa-75215020269c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:44.971960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 274/549] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:44.977829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ed3f4-99fc-451f-b9fa-75215020269c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:45.022662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:45.041068 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d1147-e18b-4544-b40c-fb4190801dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:45.041885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d1147-e18b-4544-b40c-fb4190801dc0 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:45.042307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d1147-e18b-4544-b40c-fb4190801dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:45.042628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d1147-e18b-4544-b40c-fb4190801dc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:45.069134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d1147-e18b-4544-b40c-fb4190801dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:45.070716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d1147-e18b-4544-b40c-fb4190801dc0 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-3d6d1147-e18b-4544-b40c-fb4190801dc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:45.095429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:45.096571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:45.097440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:45.097440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:45.097440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:45.097440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:45.099451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 276/550] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:45.108441 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ed3f4-99fc-451f-b9fa-75215020269c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:45.109718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d1147-e18b-4544-b40c-fb4190801dc0 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:45.163719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004ae3e-eeb5-4790-afb9-9c85c596550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:45.166456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ae3e-eeb5-4790-afb9-9c85c596550d None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:45.167233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ae3e-eeb5-4790-afb9-9c85c596550d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:45.169917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ae3e-eeb5-4790-afb9-9c85c596550d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:45.197325 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004ae3e-eeb5-4790-afb9-9c85c596550d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:45.199396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004ae3e-eeb5-4790-afb9-9c85c596550d None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:45.228619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-6004ae3e-eeb5-4790-afb9-9c85c596550d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:45.249721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:45.250675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:45.251727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:45.251727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:45.251727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:45.251727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:45.257714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 275/551] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:45.264966 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004ae3e-eeb5-4790-afb9-9c85c596550d None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:45.332065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d1147-e18b-4544-b40c-fb4190801dc0 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:45.401326 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758134d1-af63-459e-b539-d5adaecaf8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:45.402965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758134d1-af63-459e-b539-d5adaecaf8ad None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:45.404872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758134d1-af63-459e-b539-d5adaecaf8ad None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:45.404872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758134d1-af63-459e-b539-d5adaecaf8ad None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:45.405878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758134d1-af63-459e-b539-d5adaecaf8ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:45.405878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758134d1-af63-459e-b539-d5adaecaf8ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:45.407333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758134d1-af63-459e-b539-d5adaecaf8ad None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:45.442203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 277/552] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:45.455844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a7602-c044-4f42-87b8-fac7093d82f9 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:45.456120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a7602-c044-4f42-87b8-fac7093d82f9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:45.456468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a7602-c044-4f42-87b8-fac7093d82f9 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:45.460118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 278/553] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:45.460827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a7602-c044-4f42-87b8-fac7093d82f9 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:45.479350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:45.492376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004ae3e-eeb5-4790-afb9-9c85c596550d None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:45.531518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ed4c5-1851-4762-b0f1-816b6757a63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:45.532576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ed4c5-1851-4762-b0f1-816b6757a63c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:45.533075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ed4c5-1851-4762-b0f1-816b6757a63c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:45.533533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ed4c5-1851-4762-b0f1-816b6757a63c None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:45.534417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ed4c5-1851-4762-b0f1-816b6757a63c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:45.534897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ed4c5-1851-4762-b0f1-816b6757a63c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:45.536575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ed4c5-1851-4762-b0f1-816b6757a63c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:45.557667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 276/554] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:45.559314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781ccdc-267a-41af-96e3-b7631bf22f09 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:45.559805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781ccdc-267a-41af-96e3-b7631bf22f09 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:45.560281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781ccdc-267a-41af-96e3-b7631bf22f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:45.578526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-1781ccdc-267a-41af-96e3-b7631bf22f09 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:45.580137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1781ccdc-267a-41af-96e3-b7631bf22f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:45.648779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a7602-c044-4f42-87b8-fac7093d82f9 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:45.664657 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 277/555] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:45.666261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781ccdc-267a-41af-96e3-b7631bf22f09 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:45.680697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:45.750271 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c48892-571c-40a5-b432-eab5618113c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:45.750999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c48892-571c-40a5-b432-eab5618113c6 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:45.751285 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c48892-571c-40a5-b432-eab5618113c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:45.751563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c48892-571c-40a5-b432-eab5618113c6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:45.785841 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:45.793308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c48892-571c-40a5-b432-eab5618113c6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:45.794795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c48892-571c-40a5-b432-eab5618113c6 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-23c48892-571c-40a5-b432-eab5618113c6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:45.811088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:45.812217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:45.813626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:45.813626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:45.813626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:45.813626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:45.813626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:45.813626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:45.813626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:45.813626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:45.813626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:40:45.813626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:45.813626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 279/556] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:40:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:45.814210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c48892-571c-40a5-b432-eab5618113c6 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:45.842126 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781ccdc-267a-41af-96e3-b7631bf22f09 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:45.928741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c48892-571c-40a5-b432-eab5618113c6 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:45.959240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa152ee0-527f-4414-a1cc-a9b43d6b91a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:45.960030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa152ee0-527f-4414-a1cc-a9b43d6b91a7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:45.960312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa152ee0-527f-4414-a1cc-a9b43d6b91a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:45.960605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa152ee0-527f-4414-a1cc-a9b43d6b91a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:45.970809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa152ee0-527f-4414-a1cc-a9b43d6b91a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:45.972593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa152ee0-527f-4414-a1cc-a9b43d6b91a7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:45.974571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b7996-0af8-4f44-b17c-72d66a28b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:45.975907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b7996-0af8-4f44-b17c-72d66a28b283 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:45.976511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b7996-0af8-4f44-b17c-72d66a28b283 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:45.977112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b7996-0af8-4f44-b17c-72d66a28b283 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-fa152ee0-527f-4414-a1cc-a9b43d6b91a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:45.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:45.991774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:45.993203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:45.993203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:45.993203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:45.993203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:45.993203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:45.993203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:45.993203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:45.993203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:45.993203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:40:45.993203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:46.001053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 278/557] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jul 19 20:40:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:46.002196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa152ee0-527f-4414-a1cc-a9b43d6b91a7 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:46.003625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b7996-0af8-4f44-b17c-72d66a28b283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-de3b7996-0af8-4f44-b17c-72d66a28b283 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:46.018714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:46.019716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:46.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:46.025089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 280/558] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:46.026646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba2129-8960-4137-a715-89179ad796e4 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:46.026946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba2129-8960-4137-a715-89179ad796e4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:46.027231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba2129-8960-4137-a715-89179ad796e4 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:46.033204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 281/559] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:46.034084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba2129-8960-4137-a715-89179ad796e4 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:46.068729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:46.146565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa152ee0-527f-4414-a1cc-a9b43d6b91a7 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:46.168976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba2129-8960-4137-a715-89179ad796e4 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:46.194856 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec534-d91e-4d6c-97a1-306d58b7a758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:46.195643 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec534-d91e-4d6c-97a1-306d58b7a758 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:46.195939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec534-d91e-4d6c-97a1-306d58b7a758 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:46.196212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec534-d91e-4d6c-97a1-306d58b7a758 None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:46.197370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ec534-d91e-4d6c-97a1-306d58b7a758 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:46.197772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ec534-d91e-4d6c-97a1-306d58b7a758 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:46.199544 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ec534-d91e-4d6c-97a1-306d58b7a758 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:46.217874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:46.222281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdccd9-23e6-458c-8991-68b94e1e530d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:46.222958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdccd9-23e6-458c-8991-68b94e1e530d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:46.223369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdccd9-23e6-458c-8991-68b94e1e530d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:46.223624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdccd9-23e6-458c-8991-68b94e1e530d None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:46.224666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdccd9-23e6-458c-8991-68b94e1e530d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:46.225007 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdccd9-23e6-458c-8991-68b94e1e530d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:46.226427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdccd9-23e6-458c-8991-68b94e1e530d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:46.238571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 279/560] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jul 19 20:40:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:46.238571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec534-d91e-4d6c-97a1-306d58b7a758 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:46.259508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 282/561] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jul 19 20:40:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:46.259508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7344372-cde0-4dc3-ab27-49d80559c4fd None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:46.259508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7344372-cde0-4dc3-ab27-49d80559c4fd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:46.259508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7344372-cde0-4dc3-ab27-49d80559c4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:46.274434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-f7344372-cde0-4dc3-ab27-49d80559c4fd None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:46.275690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f7344372-cde0-4dc3-ab27-49d80559c4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:46.372353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 283/562] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:46.372353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7344372-cde0-4dc3-ab27-49d80559c4fd None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:46.385560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec534-d91e-4d6c-97a1-306d58b7a758 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:46.393872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:46.434022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f7e62-1fe6-403a-a920-082794564d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:46.434894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f7e62-1fe6-403a-a920-082794564d12 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:46.435375 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f7e62-1fe6-403a-a920-082794564d12 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:46.435761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f7e62-1fe6-403a-a920-082794564d12 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:46.437071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f7e62-1fe6-403a-a920-082794564d12 None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:46.437541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f7e62-1fe6-403a-a920-082794564d12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:46.439810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f7e62-1fe6-403a-a920-082794564d12 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:46.453303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 280/563] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:46.454611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f7e62-1fe6-403a-a920-082794564d12 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:46.506510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7344372-cde0-4dc3-ab27-49d80559c4fd None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:46.537675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:46.550890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf90a5b-6cea-4e5f-b6d2-0ea82939c1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:46.552163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf90a5b-6cea-4e5f-b6d2-0ea82939c1c7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:46.552524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf90a5b-6cea-4e5f-b6d2-0ea82939c1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:46.552830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf90a5b-6cea-4e5f-b6d2-0ea82939c1c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:46.586265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f7e62-1fe6-403a-a920-082794564d12 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:46.628814 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278f0bb-4adf-4305-97b9-46004034010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:46.629718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278f0bb-4adf-4305-97b9-46004034010b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:46.630120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278f0bb-4adf-4305-97b9-46004034010b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:46.630409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278f0bb-4adf-4305-97b9-46004034010b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:46.642387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf90a5b-6cea-4e5f-b6d2-0ea82939c1c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:46.644063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf90a5b-6cea-4e5f-b6d2-0ea82939c1c7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:46.647660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278f0bb-4adf-4305-97b9-46004034010b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:46.650139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278f0bb-4adf-4305-97b9-46004034010b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:46.670163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-1278f0bb-4adf-4305-97b9-46004034010b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:46.671000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:46.672494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:46.674275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:46.674275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:46.674275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:46.674275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:46.674275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 281/564] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:46.674275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278f0bb-4adf-4305-97b9-46004034010b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:46.793400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 284/565] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:46.793400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf90a5b-6cea-4e5f-b6d2-0ea82939c1c7 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:46.811038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278f0bb-4adf-4305-97b9-46004034010b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:46.831030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:46.859968 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d4e29-f9ac-4842-a1c3-934f63680ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:46.861019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d4e29-f9ac-4842-a1c3-934f63680ad5 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:46.861372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d4e29-f9ac-4842-a1c3-934f63680ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:46.861652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d4e29-f9ac-4842-a1c3-934f63680ad5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:46.873988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d4e29-f9ac-4842-a1c3-934f63680ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-bc3d4e29-f9ac-4842-a1c3-934f63680ad5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:46.881793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:46.882877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:46.883848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 282/566] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:46.884792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d4e29-f9ac-4842-a1c3-934f63680ad5 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:46.908738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf90a5b-6cea-4e5f-b6d2-0ea82939c1c7 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:46.925044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:46.951300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116391a6-8ac8-4013-a8a0-a73cdc6a2d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:46.952792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116391a6-8ac8-4013-a8a0-a73cdc6a2d69 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:46.953223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116391a6-8ac8-4013-a8a0-a73cdc6a2d69 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:46.953573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116391a6-8ac8-4013-a8a0-a73cdc6a2d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:46.969380 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116391a6-8ac8-4013-a8a0-a73cdc6a2d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:46.971411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116391a6-8ac8-4013-a8a0-a73cdc6a2d69 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-116391a6-8ac8-4013-a8a0-a73cdc6a2d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:46.994426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:46.995418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:46.996353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:46.996353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:46.996353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:46.996353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:46.996353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 285/567] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:46.997182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116391a6-8ac8-4013-a8a0-a73cdc6a2d69 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:47.005357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d4e29-f9ac-4842-a1c3-934f63680ad5 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:47.043335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b097d1-9551-4ced-a1fa-5437b10a33c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:47.044039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b097d1-9551-4ced-a1fa-5437b10a33c6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.044206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b097d1-9551-4ced-a1fa-5437b10a33c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.044507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b097d1-9551-4ced-a1fa-5437b10a33c6 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:47.045288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b097d1-9551-4ced-a1fa-5437b10a33c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:47.045526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b097d1-9551-4ced-a1fa-5437b10a33c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:47.046903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b097d1-9551-4ced-a1fa-5437b10a33c6 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:47.059899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 283/568] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:47.061339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523097b2-6392-4a95-b3a5-dbc981b69f50 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.061634 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523097b2-6392-4a95-b3a5-dbc981b69f50 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.061955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523097b2-6392-4a95-b3a5-dbc981b69f50 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:47.063182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 284/569] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:47.063914 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523097b2-6392-4a95-b3a5-dbc981b69f50 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:47.103250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:47.111729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116391a6-8ac8-4013-a8a0-a73cdc6a2d69 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:47.154565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d617b4-9063-47eb-a0a6-e9cb53f5db1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:47.155362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d617b4-9063-47eb-a0a6-e9cb53f5db1d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.155908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d617b4-9063-47eb-a0a6-e9cb53f5db1d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.156192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d617b4-9063-47eb-a0a6-e9cb53f5db1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:47.169731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d617b4-9063-47eb-a0a6-e9cb53f5db1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:47.171662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d617b4-9063-47eb-a0a6-e9cb53f5db1d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:47.182680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523097b2-6392-4a95-b3a5-dbc981b69f50 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-e5d617b4-9063-47eb-a0a6-e9cb53f5db1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:47.193651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:47.195098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:47.196287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:47.196287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:47.196287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:47.196287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:47.196287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 286/570] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:47.196287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d617b4-9063-47eb-a0a6-e9cb53f5db1d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:47.228116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:47.235068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a9b50-2f90-4a24-8a29-956119339c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:47.235785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a9b50-2f90-4a24-8a29-956119339c6e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.236068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a9b50-2f90-4a24-8a29-956119339c6e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.236365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a9b50-2f90-4a24-8a29-956119339c6e None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:47.237208 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a9b50-2f90-4a24-8a29-956119339c6e None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:47.237520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a9b50-2f90-4a24-8a29-956119339c6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:47.239207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a9b50-2f90-4a24-8a29-956119339c6e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:47.248768 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 285/571] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:47.250199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a9b50-2f90-4a24-8a29-956119339c6e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:47.331231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d617b4-9063-47eb-a0a6-e9cb53f5db1d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:47.372637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:47.378669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f176cf-3115-430f-86a9-61d5c3e29be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:47.379978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f176cf-3115-430f-86a9-61d5c3e29be3 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.380242 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f176cf-3115-430f-86a9-61d5c3e29be3 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.380669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f176cf-3115-430f-86a9-61d5c3e29be3 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:47.381418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f176cf-3115-430f-86a9-61d5c3e29be3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:47.381734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f176cf-3115-430f-86a9-61d5c3e29be3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:47.382544 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a9b50-2f90-4a24-8a29-956119339c6e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:47.383412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f176cf-3115-430f-86a9-61d5c3e29be3 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:47.399613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 287/572] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:47.400971 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d958c8-e975-464b-9486-49128e93a827 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.401265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d958c8-e975-464b-9486-49128e93a827 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.401547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d958c8-e975-464b-9486-49128e93a827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:47.418912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-99d958c8-e975-464b-9486-49128e93a827 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:47.420169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-99d958c8-e975-464b-9486-49128e93a827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:47.427323 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a95dd-4ebc-41eb-994f-1893c26e7c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:47.428197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a95dd-4ebc-41eb-994f-1893c26e7c18 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.428515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a95dd-4ebc-41eb-994f-1893c26e7c18 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.428800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a95dd-4ebc-41eb-994f-1893c26e7c18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:47.437142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a95dd-4ebc-41eb-994f-1893c26e7c18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:47.438609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a95dd-4ebc-41eb-994f-1893c26e7c18 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-825a95dd-4ebc-41eb-994f-1893c26e7c18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:47.454885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:47.456667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:47.458874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:47.458874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:47.458874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:47.458874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:47.458874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:47.458874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:47.458874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:47.458874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:47.458874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 286/573] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:47.458874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a95dd-4ebc-41eb-994f-1893c26e7c18 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:47.508187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 288/574] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:47.509326 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d958c8-e975-464b-9486-49128e93a827 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:47.526297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:47.575244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a95dd-4ebc-41eb-994f-1893c26e7c18 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:47.622094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54a3ca-a5f1-49fb-aebe-e34e3cd75620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:47.622680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54a3ca-a5f1-49fb-aebe-e34e3cd75620 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.622990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54a3ca-a5f1-49fb-aebe-e34e3cd75620 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.623261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54a3ca-a5f1-49fb-aebe-e34e3cd75620 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:47.624168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54a3ca-a5f1-49fb-aebe-e34e3cd75620 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:47.624612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54a3ca-a5f1-49fb-aebe-e34e3cd75620 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:47.625935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54a3ca-a5f1-49fb-aebe-e34e3cd75620 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:47.639625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d958c8-e975-464b-9486-49128e93a827 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:47.642421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 287/575] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:47.643353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54a3ca-a5f1-49fb-aebe-e34e3cd75620 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:47.660637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:47.678221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de88feb-1d04-41dc-947e-81d79ee69b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:47.679018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de88feb-1d04-41dc-947e-81d79ee69b67 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.679300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de88feb-1d04-41dc-947e-81d79ee69b67 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.679573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de88feb-1d04-41dc-947e-81d79ee69b67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:47.691925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de88feb-1d04-41dc-947e-81d79ee69b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:47.694290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de88feb-1d04-41dc-947e-81d79ee69b67 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-6de88feb-1d04-41dc-947e-81d79ee69b67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:47.715941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:47.717127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:47.718195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:47.718195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:47.718195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:47.718195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:47.718195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 289/576] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:47.718541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de88feb-1d04-41dc-947e-81d79ee69b67 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:47.761129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54a3ca-a5f1-49fb-aebe-e34e3cd75620 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:47.785509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:47.801224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db08e6e-ff53-4320-b62a-094b289fc35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:47.804336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db08e6e-ff53-4320-b62a-094b289fc35c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.804336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db08e6e-ff53-4320-b62a-094b289fc35c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.804336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db08e6e-ff53-4320-b62a-094b289fc35c None admin] PATH_INFO: `/v3/users/glance` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:47.813082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db08e6e-ff53-4320-b62a-094b289fc35c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:47.814936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db08e6e-ff53-4320-b62a-094b289fc35c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-9db08e6e-ff53-4320-b62a-094b289fc35c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:47.830898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:47.832185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:47.833216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:47.833216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:47.833216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:47.833216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:47.833216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:47.833216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:47.833216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:47.833216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:47.833216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:40:47.833216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:47.833216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 288/577] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:47.833841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de88feb-1d04-41dc-947e-81d79ee69b67 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:47.833917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db08e6e-ff53-4320-b62a-094b289fc35c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:47.878293 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfa208-8d5d-48dc-90a2-d01a6d9bea12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:47.879304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfa208-8d5d-48dc-90a2-d01a6d9bea12 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.879759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfa208-8d5d-48dc-90a2-d01a6d9bea12 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.880249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfa208-8d5d-48dc-90a2-d01a6d9bea12 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:47.881369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfa208-8d5d-48dc-90a2-d01a6d9bea12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:47.881879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfa208-8d5d-48dc-90a2-d01a6d9bea12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:47.884524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfa208-8d5d-48dc-90a2-d01a6d9bea12 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:47.903440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 290/578] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:47.904857 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfa208-8d5d-48dc-90a2-d01a6d9bea12 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:47.919927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:47.951073 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db08e6e-ff53-4320-b62a-094b289fc35c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:47.991139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad88435-7215-42ea-82fd-359b1209e1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:47.991948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad88435-7215-42ea-82fd-359b1209e1b6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:47.992228 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad88435-7215-42ea-82fd-359b1209e1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:47.992533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad88435-7215-42ea-82fd-359b1209e1b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:48.005942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad88435-7215-42ea-82fd-359b1209e1b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-9ad88435-7215-42ea-82fd-359b1209e1b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:48.014457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:48.015624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 289/579] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:48.016628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad88435-7215-42ea-82fd-359b1209e1b6 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:48.036770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfa208-8d5d-48dc-90a2-d01a6d9bea12 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:48.061587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:48.075913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dc5e7-1305-462f-bd8d-0f9029861e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:48.076744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dc5e7-1305-462f-bd8d-0f9029861e2a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:48.077033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dc5e7-1305-462f-bd8d-0f9029861e2a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:48.077312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dc5e7-1305-462f-bd8d-0f9029861e2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:48.084063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7dc5e7-1305-462f-bd8d-0f9029861e2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:48.085604 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7dc5e7-1305-462f-bd8d-0f9029861e2a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-3b7dc5e7-1305-462f-bd8d-0f9029861e2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:48.100668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:48.101712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:48.102812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:48.102812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:48.102812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:48.102812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:48.102812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:48.102812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:48.102812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:48.102812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:48.102812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 291/580] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:48.102812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dc5e7-1305-462f-bd8d-0f9029861e2a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:48.127533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad88435-7215-42ea-82fd-359b1209e1b6 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:48.161719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3a954-d4a4-492d-8322-9d70f7402151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:48.162286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3a954-d4a4-492d-8322-9d70f7402151 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:48.162569 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3a954-d4a4-492d-8322-9d70f7402151 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:48.162864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3a954-d4a4-492d-8322-9d70f7402151 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:48.163555 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3a954-d4a4-492d-8322-9d70f7402151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:48.163865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3a954-d4a4-492d-8322-9d70f7402151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:48.165264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3a954-d4a4-492d-8322-9d70f7402151 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:48.166681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:48.177601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 290/581] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:48.178398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3a954-d4a4-492d-8322-9d70f7402151 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:48.212880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dc5e7-1305-462f-bd8d-0f9029861e2a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:48.247988 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebc422-aebd-4d34-afe1-21b5d5081577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:48.248694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebc422-aebd-4d34-afe1-21b5d5081577 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:48.248983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebc422-aebd-4d34-afe1-21b5d5081577 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:48.249259 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebc422-aebd-4d34-afe1-21b5d5081577 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:48.250097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebc422-aebd-4d34-afe1-21b5d5081577 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:48.250407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebc422-aebd-4d34-afe1-21b5d5081577 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:48.251832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebc422-aebd-4d34-afe1-21b5d5081577 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:48.269389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 292/582] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jul 19 20:40:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:48.270596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebc422-aebd-4d34-afe1-21b5d5081577 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:48.281316 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3a954-d4a4-492d-8322-9d70f7402151 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:48.312120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:48.323439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aed80e-f83b-4065-8012-d8be9f5976b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:48.324509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aed80e-f83b-4065-8012-d8be9f5976b6 None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:48.324798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aed80e-f83b-4065-8012-d8be9f5976b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:48.325089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aed80e-f83b-4065-8012-d8be9f5976b6 None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/928b8b2b510b4c939b37b2c0dfa57060/roles/b2e0a9d2d13844e980a69f441f36d492` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:48.341317 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aed80e-f83b-4065-8012-d8be9f5976b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=928b8b2b510b4c939b37b2c0dfa57060, role_id=b2e0a9d2d13844e980a69f441f36d492)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:48.342771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aed80e-f83b-4065-8012-d8be9f5976b6 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:48.356403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 291/583] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:40:48 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/928b8b2b510b4c939b37b2c0dfa57060/roles/b2e0a9d2d13844e980a69f441f36d492 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:40:48.357555 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aed80e-f83b-4065-8012-d8be9f5976b6 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:48.396539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebc422-aebd-4d34-afe1-21b5d5081577 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:48.431856 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:48.448500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdc3c0-eca1-4d39-8cb9-db9c79bb72fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:48.449191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdc3c0-eca1-4d39-8cb9-db9c79bb72fe None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:48.449475 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdc3c0-eca1-4d39-8cb9-db9c79bb72fe None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:48.449753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdc3c0-eca1-4d39-8cb9-db9c79bb72fe None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:48.450547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdc3c0-eca1-4d39-8cb9-db9c79bb72fe None admin] RBAC: Adding query filter params (name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:48.450857 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdc3c0-eca1-4d39-8cb9-db9c79bb72fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:48.452246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdc3c0-eca1-4d39-8cb9-db9c79bb72fe None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:48.465858 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 293/584] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:48.465858 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdc3c0-eca1-4d39-8cb9-db9c79bb72fe None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:48.504447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aed80e-f83b-4065-8012-d8be9f5976b6 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:48.547948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e295a-71d4-4472-9527-95753c6cef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:48.548732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e295a-71d4-4472-9527-95753c6cef7f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:48.549020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e295a-71d4-4472-9527-95753c6cef7f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:48.549297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e295a-71d4-4472-9527-95753c6cef7f None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:48.550086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e295a-71d4-4472-9527-95753c6cef7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:48.550397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e295a-71d4-4472-9527-95753c6cef7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:48.552416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e295a-71d4-4472-9527-95753c6cef7f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:48.564673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 292/585] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:48.565469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e295a-71d4-4472-9527-95753c6cef7f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:48.575200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:48.583500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdc3c0-eca1-4d39-8cb9-db9c79bb72fe None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:48.627440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a5139-0f7f-4b20-8177-366072b865e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:48.628249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a5139-0f7f-4b20-8177-366072b865e7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:48.629136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a5139-0f7f-4b20-8177-366072b865e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:48.629445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a5139-0f7f-4b20-8177-366072b865e7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:48.642751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a5139-0f7f-4b20-8177-366072b865e7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:48.644269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a5139-0f7f-4b20-8177-366072b865e7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-033a5139-0f7f-4b20-8177-366072b865e7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:48.660376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:48.662188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:48.663336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:48.663336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:48.663336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:48.663336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:48.663336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:48.663336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:48.663336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:48.663336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:48.663336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:40:48.663336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:48.663336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 294/586] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:48.663940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a5139-0f7f-4b20-8177-366072b865e7 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:48.693529 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e295a-71d4-4472-9527-95753c6cef7f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:48.696826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:48.741377 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e2688-3ae4-417e-891c-e184ebe0233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:48.742138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e2688-3ae4-417e-891c-e184ebe0233a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:48.742439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e2688-3ae4-417e-891c-e184ebe0233a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:48.742695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e2688-3ae4-417e-891c-e184ebe0233a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:48.754898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e2688-3ae4-417e-891c-e184ebe0233a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:48.756597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e2688-3ae4-417e-891c-e184ebe0233a None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-5c4e2688-3ae4-417e-891c-e184ebe0233a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:48.776049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:48.777567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:48.778735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:48.778735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:48.778735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:48.778735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:48.778735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 293/587] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:48.778735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e2688-3ae4-417e-891c-e184ebe0233a None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:48.796273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a5139-0f7f-4b20-8177-366072b865e7 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:48.840243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd18b0-1a8e-4c5a-a70f-61487445f2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:48.841073 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd18b0-1a8e-4c5a-a70f-61487445f2f0 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:48.841357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd18b0-1a8e-4c5a-a70f-61487445f2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:48.841634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd18b0-1a8e-4c5a-a70f-61487445f2f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:48.854742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd18b0-1a8e-4c5a-a70f-61487445f2f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:48.856198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd18b0-1a8e-4c5a-a70f-61487445f2f0 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-47fd18b0-1a8e-4c5a-a70f-61487445f2f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:48.877778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:48.879180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:48.880797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:48.880797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:48.880797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:48.880797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:48.883654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 295/588] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:48.884618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd18b0-1a8e-4c5a-a70f-61487445f2f0 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:48.905522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:48.911382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e2688-3ae4-417e-891c-e184ebe0233a None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:48.958144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc92e3f-ffd0-417c-a1c9-215ed006f5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:48.958713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc92e3f-ffd0-417c-a1c9-215ed006f5d2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:48.958993 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc92e3f-ffd0-417c-a1c9-215ed006f5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:48.959269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc92e3f-ffd0-417c-a1c9-215ed006f5d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:48.960106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc92e3f-ffd0-417c-a1c9-215ed006f5d2 None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=812342d822f54aaf98e324d5d4e56753) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:48.960443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc92e3f-ffd0-417c-a1c9-215ed006f5d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:48.961876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc92e3f-ffd0-417c-a1c9-215ed006f5d2 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:48.972362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 294/589] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:48 2022] GET /identity/v3/role_assignments?user.id=812342d822f54aaf98e324d5d4e56753&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:48.972362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc92e3f-ffd0-417c-a1c9-215ed006f5d2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:49.000482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:49.030361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd18b0-1a8e-4c5a-a70f-61487445f2f0 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:49.072237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866aa0b7-c834-48aa-b53c-0d77e0d562d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:49.073174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866aa0b7-c834-48aa-b53c-0d77e0d562d1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:49.073463 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866aa0b7-c834-48aa-b53c-0d77e0d562d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:49.073737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866aa0b7-c834-48aa-b53c-0d77e0d562d1 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:49.074602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866aa0b7-c834-48aa-b53c-0d77e0d562d1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:49.074914 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866aa0b7-c834-48aa-b53c-0d77e0d562d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:49.076423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866aa0b7-c834-48aa-b53c-0d77e0d562d1 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:49.106238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 296/590] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jul 19 20:40:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:49.106238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866aa0b7-c834-48aa-b53c-0d77e0d562d1 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:49.127859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc92e3f-ffd0-417c-a1c9-215ed006f5d2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:49.180214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956404a-1323-4733-bde6-909e8ba0af73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:49.180931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956404a-1323-4733-bde6-909e8ba0af73 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:49.182163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956404a-1323-4733-bde6-909e8ba0af73 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:49.182163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956404a-1323-4733-bde6-909e8ba0af73 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:49.182752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956404a-1323-4733-bde6-909e8ba0af73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:49.184301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956404a-1323-4733-bde6-909e8ba0af73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:49.184906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956404a-1323-4733-bde6-909e8ba0af73 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:49.204628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 295/591] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:49.205778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956404a-1323-4733-bde6-909e8ba0af73 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:49.228683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:49.256576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866aa0b7-c834-48aa-b53c-0d77e0d562d1 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:49.294644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e80c14-0dfb-4c67-a17f-479c16a8d76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:49.295795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e80c14-0dfb-4c67-a17f-479c16a8d76b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:49.296082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e80c14-0dfb-4c67-a17f-479c16a8d76b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:49.296384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e80c14-0dfb-4c67-a17f-479c16a8d76b None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:49.297125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e80c14-0dfb-4c67-a17f-479c16a8d76b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:49.297434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e80c14-0dfb-4c67-a17f-479c16a8d76b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:49.299521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e80c14-0dfb-4c67-a17f-479c16a8d76b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:49.302423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:49.318783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 297/592] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:49.318783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e80c14-0dfb-4c67-a17f-479c16a8d76b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:49.331844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956404a-1323-4733-bde6-909e8ba0af73 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:49.372642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674f4bb-2522-449e-bef1-c4989d37101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:49.374202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674f4bb-2522-449e-bef1-c4989d37101c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:49.374811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674f4bb-2522-449e-bef1-c4989d37101c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:49.375874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674f4bb-2522-449e-bef1-c4989d37101c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:49.396235 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674f4bb-2522-449e-bef1-c4989d37101c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:49.398911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674f4bb-2522-449e-bef1-c4989d37101c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-0674f4bb-2522-449e-bef1-c4989d37101c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:49.425017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:49.426001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:49.426936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:49.426936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:49.426936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:49.426936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:49.434557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 296/593] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:49.434557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674f4bb-2522-449e-bef1-c4989d37101c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:49.445976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e80c14-0dfb-4c67-a17f-479c16a8d76b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:49.482763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:49.489545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0e157-ed66-453f-b67d-3a505ed48f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:49.490387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0e157-ed66-453f-b67d-3a505ed48f66 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:49.490785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0e157-ed66-453f-b67d-3a505ed48f66 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:49.491110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0e157-ed66-453f-b67d-3a505ed48f66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:49.499346 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0e157-ed66-453f-b67d-3a505ed48f66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:49.501045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0e157-ed66-453f-b67d-3a505ed48f66 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-6ad0e157-ed66-453f-b67d-3a505ed48f66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:49.523347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:49.524301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:49.525243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:49.525243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:49.525243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:49.525243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:49.525243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:49.525243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:49.525243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:49.525243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:49.530617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 298/594] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:49.530617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0e157-ed66-453f-b67d-3a505ed48f66 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:49.578756 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674f4bb-2522-449e-bef1-c4989d37101c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:49.614324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:49.622169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f010c-807c-4ecc-bbc9-7614bf0ec70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:49.623280 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f010c-807c-4ecc-bbc9-7614bf0ec70a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:49.623745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f010c-807c-4ecc-bbc9-7614bf0ec70a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:49.624300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f010c-807c-4ecc-bbc9-7614bf0ec70a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:49.633916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f010c-807c-4ecc-bbc9-7614bf0ec70a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:49.636206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f010c-807c-4ecc-bbc9-7614bf0ec70a None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-222f010c-807c-4ecc-bbc9-7614bf0ec70a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:49.653735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:49.655517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:49.656620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:49.656620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:49.656620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:49.656620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:49.656620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:49.656620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:49.656620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:49.656620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:49.656620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:40:49.656620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:49.658720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 297/595] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:49.658955 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0e157-ed66-453f-b67d-3a505ed48f66 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:49.660096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f010c-807c-4ecc-bbc9-7614bf0ec70a None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:49.712768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa1c25-f568-475c-afac-992b197bdad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:49.713391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa1c25-f568-475c-afac-992b197bdad7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:49.713660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa1c25-f568-475c-afac-992b197bdad7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:49.713933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa1c25-f568-475c-afac-992b197bdad7 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:49.714843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa1c25-f568-475c-afac-992b197bdad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:49.715152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa1c25-f568-475c-afac-992b197bdad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:49.716788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa1c25-f568-475c-afac-992b197bdad7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:49.736285 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 299/596] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:49.737253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa1c25-f568-475c-afac-992b197bdad7 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:49.782138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f010c-807c-4ecc-bbc9-7614bf0ec70a None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:49.795318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:49.827468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3a963-76f8-4b91-a8b8-6155a251eb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:49.828688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3a963-76f8-4b91-a8b8-6155a251eb16 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:49.829184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3a963-76f8-4b91-a8b8-6155a251eb16 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:49.829780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3a963-76f8-4b91-a8b8-6155a251eb16 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:49.830894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3a963-76f8-4b91-a8b8-6155a251eb16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:49.831411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3a963-76f8-4b91-a8b8-6155a251eb16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:49.833692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3a963-76f8-4b91-a8b8-6155a251eb16 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:49.848483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 298/597] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:49.853811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa1c25-f568-475c-afac-992b197bdad7 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:49.855477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3a963-76f8-4b91-a8b8-6155a251eb16 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:49.893228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e50c55-cee1-49d3-86cd-dcb2a854c976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:49.893897 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e50c55-cee1-49d3-86cd-dcb2a854c976 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:49.894177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e50c55-cee1-49d3-86cd-dcb2a854c976 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:49.894454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e50c55-cee1-49d3-86cd-dcb2a854c976 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:49.895333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e50c55-cee1-49d3-86cd-dcb2a854c976 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:49.895644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e50c55-cee1-49d3-86cd-dcb2a854c976 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:49.897095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e50c55-cee1-49d3-86cd-dcb2a854c976 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:49.916243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 300/598] 10.222.0.85 () {56 vars in 1139 bytes} [Tue Jul 19 20:40:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:49.917181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e50c55-cee1-49d3-86cd-dcb2a854c976 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:49.919808 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:49.975982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3a963-76f8-4b91-a8b8-6155a251eb16 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:50.017062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af13e0-2fed-41bc-8ac3-fd43553f2d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:50.018193 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af13e0-2fed-41bc-8ac3-fd43553f2d5e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:50.018701 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af13e0-2fed-41bc-8ac3-fd43553f2d5e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:50.019182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af13e0-2fed-41bc-8ac3-fd43553f2d5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:50.027585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af13e0-2fed-41bc-8ac3-fd43553f2d5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:50.029183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e50c55-cee1-49d3-86cd-dcb2a854c976 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:50.029564 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af13e0-2fed-41bc-8ac3-fd43553f2d5e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-11af13e0-2fed-41bc-8ac3-fd43553f2d5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:50.045243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:50.046226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:50.047129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:50.047129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:50.047129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:50.047129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:50.047129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:50.047129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:50.047129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:50.047129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:50.048748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:50.051133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 299/599] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:50.052902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af13e0-2fed-41bc-8ac3-fd43553f2d5e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:50.082091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83703b-30d9-49ed-be06-b36c278bcd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:50.083501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83703b-30d9-49ed-be06-b36c278bcd88 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:50.083927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83703b-30d9-49ed-be06-b36c278bcd88 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:50.084425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83703b-30d9-49ed-be06-b36c278bcd88 None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/e35e7720e3bf4c9e80bdefa6a1cb8296/roles/b2e0a9d2d13844e980a69f441f36d492` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:50.107145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83703b-30d9-49ed-be06-b36c278bcd88 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=e35e7720e3bf4c9e80bdefa6a1cb8296, role_id=b2e0a9d2d13844e980a69f441f36d492)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:50.109437 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83703b-30d9-49ed-be06-b36c278bcd88 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:50.128918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 301/600] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:40:49 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/e35e7720e3bf4c9e80bdefa6a1cb8296/roles/b2e0a9d2d13844e980a69f441f36d492 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:40:50.130110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83703b-30d9-49ed-be06-b36c278bcd88 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:50.179676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af13e0-2fed-41bc-8ac3-fd43553f2d5e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:50.215884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:50.281364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72550b-8c84-4265-8aac-b1ef87648d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:50.282406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72550b-8c84-4265-8aac-b1ef87648d53 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:50.282825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72550b-8c84-4265-8aac-b1ef87648d53 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:50.283225 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72550b-8c84-4265-8aac-b1ef87648d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:50.301889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72550b-8c84-4265-8aac-b1ef87648d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:50.305419 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72550b-8c84-4265-8aac-b1ef87648d53 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-2c72550b-8c84-4265-8aac-b1ef87648d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:50.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:50.342449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:50.343695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:50.343695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:50.343695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:50.343695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:50.346840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 300/601] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:50.355202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72550b-8c84-4265-8aac-b1ef87648d53 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:50.402133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83703b-30d9-49ed-be06-b36c278bcd88 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:50.448321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7e974-179d-4233-a04e-c00ececa5e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:50.449341 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7e974-179d-4233-a04e-c00ececa5e40 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:50.449751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7e974-179d-4233-a04e-c00ececa5e40 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:50.450155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7e974-179d-4233-a04e-c00ececa5e40 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:50.451188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7e974-179d-4233-a04e-c00ececa5e40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:50.451723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7e974-179d-4233-a04e-c00ececa5e40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:50.453238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7e974-179d-4233-a04e-c00ececa5e40 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:50.469325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 302/602] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:50.477042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7e974-179d-4233-a04e-c00ececa5e40 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:50.504761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:50.523611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72550b-8c84-4265-8aac-b1ef87648d53 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:50.567019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97a482-4349-42b8-bee7-f433d25d1026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:50.568221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97a482-4349-42b8-bee7-f433d25d1026 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:50.568942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97a482-4349-42b8-bee7-f433d25d1026 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:50.569636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97a482-4349-42b8-bee7-f433d25d1026 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:50.571084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97a482-4349-42b8-bee7-f433d25d1026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:50.571788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97a482-4349-42b8-bee7-f433d25d1026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:50.574452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97a482-4349-42b8-bee7-f433d25d1026 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:50.594482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 301/603] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:50.597189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2381d-be0f-4d99-91b9-f0bc6f745868 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:50.597832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2381d-be0f-4d99-91b9-f0bc6f745868 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:50.598837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2381d-be0f-4d99-91b9-f0bc6f745868 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:50.601531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 302/604] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:50.602920 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2381d-be0f-4d99-91b9-f0bc6f745868 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:50.618815 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7e974-179d-4233-a04e-c00ececa5e40 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:50.635511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:50.669722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc0b62-a86a-40a6-b21f-fa37160bb39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:50.671038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc0b62-a86a-40a6-b21f-fa37160bb39e None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:50.671613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc0b62-a86a-40a6-b21f-fa37160bb39e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:50.671915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc0b62-a86a-40a6-b21f-fa37160bb39e None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/81411204bc9f4210880d807b41da9643/roles/b2e0a9d2d13844e980a69f441f36d492` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:50.692802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc0b62-a86a-40a6-b21f-fa37160bb39e None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=81411204bc9f4210880d807b41da9643, role_id=b2e0a9d2d13844e980a69f441f36d492)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:50.694203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc0b62-a86a-40a6-b21f-fa37160bb39e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:50.715419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 303/605] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:40:50 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/81411204bc9f4210880d807b41da9643/roles/b2e0a9d2d13844e980a69f441f36d492 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:40:50.727623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67e7be-729c-4ec3-bf5a-1ba2944bd7d9 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:50.727960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67e7be-729c-4ec3-bf5a-1ba2944bd7d9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:50.728379 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67e7be-729c-4ec3-bf5a-1ba2944bd7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:50.744652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2381d-be0f-4d99-91b9-f0bc6f745868 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:50.773773 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-bf67e7be-729c-4ec3-bf5a-1ba2944bd7d9 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:50.775245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-bf67e7be-729c-4ec3-bf5a-1ba2944bd7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:50.801724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b10c0-911d-4b56-bce0-2cb20f8ebbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:50.803292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b10c0-911d-4b56-bce0-2cb20f8ebbf1 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:50.804704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b10c0-911d-4b56-bce0-2cb20f8ebbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:50.805400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b10c0-911d-4b56-bce0-2cb20f8ebbf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:50.823995 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b10c0-911d-4b56-bce0-2cb20f8ebbf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:50.825999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b10c0-911d-4b56-bce0-2cb20f8ebbf1 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-820b10c0-911d-4b56-bce0-2cb20f8ebbf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:50.854950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:50.856030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:50.857119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:50.857119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:50.857119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:50.857119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:50.857119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:50.857119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:50.857119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:50.857119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:50.860522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 303/606] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:50.867850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b10c0-911d-4b56-bce0-2cb20f8ebbf1 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:50.920734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:50.931275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 304/607] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:50.972422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67e7be-729c-4ec3-bf5a-1ba2944bd7d9 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:50.996725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:51.016480 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b10c0-911d-4b56-bce0-2cb20f8ebbf1 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:51.077769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781635f-f228-458e-b324-fa5d1cd92aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:51.078957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781635f-f228-458e-b324-fa5d1cd92aff None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:51.080351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781635f-f228-458e-b324-fa5d1cd92aff None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:51.081041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781635f-f228-458e-b324-fa5d1cd92aff None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:51.083961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781635f-f228-458e-b324-fa5d1cd92aff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:51.084752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781635f-f228-458e-b324-fa5d1cd92aff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:51.087355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781635f-f228-458e-b324-fa5d1cd92aff None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:51.109913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 304/608] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:51.117376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781635f-f228-458e-b324-fa5d1cd92aff None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:51.179461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67e7be-729c-4ec3-bf5a-1ba2944bd7d9 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:51.290563 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781635f-f228-458e-b324-fa5d1cd92aff None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:51.297723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b69b8-a50b-4994-8bf1-8e2eee81a1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:51.298672 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b69b8-a50b-4994-8bf1-8e2eee81a1d4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:51.299170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b69b8-a50b-4994-8bf1-8e2eee81a1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:51.299635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b69b8-a50b-4994-8bf1-8e2eee81a1d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:51.332846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b69b8-a50b-4994-8bf1-8e2eee81a1d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:51.342521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf95a8e-27e3-48f6-931b-ca575ca966aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:51.343424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf95a8e-27e3-48f6-931b-ca575ca966aa None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:51.343698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf95a8e-27e3-48f6-931b-ca575ca966aa None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:51.343977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf95a8e-27e3-48f6-931b-ca575ca966aa None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/5db827b3a66843749e04376559aa27c2/roles/b2e0a9d2d13844e980a69f441f36d492` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-d97b69b8-a50b-4994-8bf1-8e2eee81a1d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:51.353105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:51.354169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:51.355066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:51.360689 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 305/609] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:51.366200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b69b8-a50b-4994-8bf1-8e2eee81a1d4 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:51.376440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf95a8e-27e3-48f6-931b-ca575ca966aa None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=5db827b3a66843749e04376559aa27c2, role_id=b2e0a9d2d13844e980a69f441f36d492)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:51.377987 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf95a8e-27e3-48f6-931b-ca575ca966aa None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:51.386352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:51.414580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 305/610] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:40:51 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/5db827b3a66843749e04376559aa27c2/roles/b2e0a9d2d13844e980a69f441f36d492 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:40:51.468198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:51.508731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65ef8b-4695-48e1-be49-aa51ce6b8ce8 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:51.509141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65ef8b-4695-48e1-be49-aa51ce6b8ce8 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:51.509436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65ef8b-4695-48e1-be49-aa51ce6b8ce8 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:51.511284 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 306/611] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:51.517583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e2d20-ec88-4c4f-b012-21d915b2c214 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:51.517858 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e2d20-ec88-4c4f-b012-21d915b2c214 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:51.518135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e2d20-ec88-4c4f-b012-21d915b2c214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:51.538215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b69b8-a50b-4994-8bf1-8e2eee81a1d4 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:51.581384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83393ac9-b407-4e04-94e8-b8c725a8dd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:51.582081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83393ac9-b407-4e04-94e8-b8c725a8dd92 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:51.582367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83393ac9-b407-4e04-94e8-b8c725a8dd92 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:51.582683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83393ac9-b407-4e04-94e8-b8c725a8dd92 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:51.583507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83393ac9-b407-4e04-94e8-b8c725a8dd92 None admin] RBAC: Adding query filter params (name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:51.583759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83393ac9-b407-4e04-94e8-b8c725a8dd92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:51.585195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83393ac9-b407-4e04-94e8-b8c725a8dd92 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:51.594053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d14e2d20-ec88-4c4f-b012-21d915b2c214 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:51.594957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 306/612] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:51.596171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d14e2d20-ec88-4c4f-b012-21d915b2c214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:51.601113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83393ac9-b407-4e04-94e8-b8c725a8dd92 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:51.694672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 307/613] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:51.738755 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e2d20-ec88-4c4f-b012-21d915b2c214 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:51.775477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83393ac9-b407-4e04-94e8-b8c725a8dd92 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:51.814689 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:51.846096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a38926-7d52-4330-802c-6d3cd358fc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:51.846813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a38926-7d52-4330-802c-6d3cd358fc97 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:51.847103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a38926-7d52-4330-802c-6d3cd358fc97 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:51.847379 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a38926-7d52-4330-802c-6d3cd358fc97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:51.870351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a38926-7d52-4330-802c-6d3cd358fc97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:51.872168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a38926-7d52-4330-802c-6d3cd358fc97 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-c7a38926-7d52-4330-802c-6d3cd358fc97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:51.908232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:51.909756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:51.911806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:51.911806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:51.911806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:51.911806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:51.912972 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 307/614] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:51.915709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a38926-7d52-4330-802c-6d3cd358fc97 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:51.974615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e2d20-ec88-4c4f-b012-21d915b2c214 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:52.028740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:52.050149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8991566-f371-4caa-94b7-80c974508c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:52.050976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8991566-f371-4caa-94b7-80c974508c00 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:52.051780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8991566-f371-4caa-94b7-80c974508c00 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:52.052167 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8991566-f371-4caa-94b7-80c974508c00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:52.083185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8991566-f371-4caa-94b7-80c974508c00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-b8991566-f371-4caa-94b7-80c974508c00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:52.093443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:52.094450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:52.095305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 308/615] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:52.101423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8991566-f371-4caa-94b7-80c974508c00 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:52.139888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a38926-7d52-4330-802c-6d3cd358fc97 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:52.189327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073091a-4ca0-496c-b522-6319f5bd5b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:52.190391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073091a-4ca0-496c-b522-6319f5bd5b4f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:52.190883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073091a-4ca0-496c-b522-6319f5bd5b4f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:52.191360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073091a-4ca0-496c-b522-6319f5bd5b4f None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:52.192903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073091a-4ca0-496c-b522-6319f5bd5b4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:52.193488 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073091a-4ca0-496c-b522-6319f5bd5b4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:52.195087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073091a-4ca0-496c-b522-6319f5bd5b4f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:52.227314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 308/616] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:52.229211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073091a-4ca0-496c-b522-6319f5bd5b4f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:52.302536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8991566-f371-4caa-94b7-80c974508c00 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:52.328890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:52.382857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a065a-bd06-48eb-9eae-f654f424a3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:52.383441 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a065a-bd06-48eb-9eae-f654f424a3c2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:52.383729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a065a-bd06-48eb-9eae-f654f424a3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:52.384010 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a065a-bd06-48eb-9eae-f654f424a3c2 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:52.384811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a065a-bd06-48eb-9eae-f654f424a3c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:52.385120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a065a-bd06-48eb-9eae-f654f424a3c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:52.386541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a065a-bd06-48eb-9eae-f654f424a3c2 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:52.401571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 309/617] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:52.407606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a065a-bd06-48eb-9eae-f654f424a3c2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:52.536259 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073091a-4ca0-496c-b522-6319f5bd5b4f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:52.544966 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:52.616902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccc61f-c456-40f1-9a1e-864acb38ff10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:52.623954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccc61f-c456-40f1-9a1e-864acb38ff10 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:52.624762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccc61f-c456-40f1-9a1e-864acb38ff10 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:52.625375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccc61f-c456-40f1-9a1e-864acb38ff10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:52.631546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a065a-bd06-48eb-9eae-f654f424a3c2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:52.645733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccc61f-c456-40f1-9a1e-864acb38ff10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:52.649267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ccc61f-c456-40f1-9a1e-864acb38ff10 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-d3ccc61f-c456-40f1-9a1e-864acb38ff10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:52.683666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:52.684709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:52.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:52.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:52.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:52.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:52.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:52.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:52.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:52.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:52.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:40:52.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:52.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 309/618] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jul 19 20:40:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:52.693042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccc61f-c456-40f1-9a1e-864acb38ff10 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:52.698961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c32421-d729-4aa9-959b-10afc93b0752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:52.700107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c32421-d729-4aa9-959b-10afc93b0752 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:52.700643 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c32421-d729-4aa9-959b-10afc93b0752 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:52.701130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c32421-d729-4aa9-959b-10afc93b0752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:52.725150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c32421-d729-4aa9-959b-10afc93b0752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:52.727088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c32421-d729-4aa9-959b-10afc93b0752 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-20c32421-d729-4aa9-959b-10afc93b0752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:52.761054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:52.762267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:52.763301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:52.763301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:52.763301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:52.763301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:52.773032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 310/619] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:52.778671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c32421-d729-4aa9-959b-10afc93b0752 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:52.861321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:52.872378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccc61f-c456-40f1-9a1e-864acb38ff10 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:52.913712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb84167-9371-47d2-ada7-c9ee37dcdb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:52.914720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb84167-9371-47d2-ada7-c9ee37dcdb5d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:52.915221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb84167-9371-47d2-ada7-c9ee37dcdb5d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:52.915699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb84167-9371-47d2-ada7-c9ee37dcdb5d None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:52.916948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb84167-9371-47d2-ada7-c9ee37dcdb5d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:52.917462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb84167-9371-47d2-ada7-c9ee37dcdb5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:52.919097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb84167-9371-47d2-ada7-c9ee37dcdb5d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:52.941519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 310/620] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jul 19 20:40:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:52.948414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb84167-9371-47d2-ada7-c9ee37dcdb5d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:53.070835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb84167-9371-47d2-ada7-c9ee37dcdb5d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:53.110287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00481e-c2a7-41a0-97d0-a1ea0e039fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:53.111367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00481e-c2a7-41a0-97d0-a1ea0e039fbe None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.111958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00481e-c2a7-41a0-97d0-a1ea0e039fbe None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.112484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00481e-c2a7-41a0-97d0-a1ea0e039fbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.120188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:53.132308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c32421-d729-4aa9-959b-10afc93b0752 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:53.148303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:53.177937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938a19a-0cf0-452a-a054-14ebef87c67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:53.178608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938a19a-0cf0-452a-a054-14ebef87c67b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.178892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938a19a-0cf0-452a-a054-14ebef87c67b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.179207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938a19a-0cf0-452a-a054-14ebef87c67b None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.179881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938a19a-0cf0-452a-a054-14ebef87c67b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:53.180238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938a19a-0cf0-452a-a054-14ebef87c67b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:53.182002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938a19a-0cf0-452a-a054-14ebef87c67b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:53.199000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 311/621] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:53.200615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f6cc-6fab-4cb0-a12d-8e9a6152df20 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.200919 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f6cc-6fab-4cb0-a12d-8e9a6152df20 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.201199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f6cc-6fab-4cb0-a12d-8e9a6152df20 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.205877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00481e-c2a7-41a0-97d0-a1ea0e039fbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:53.212360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 312/622] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:53.212360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907f6cc-6fab-4cb0-a12d-8e9a6152df20 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:53.212613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00481e-c2a7-41a0-97d0-a1ea0e039fbe None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:53.320943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907f6cc-6fab-4cb0-a12d-8e9a6152df20 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:53.342854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 311/623] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:53.345145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc15f28-0ab6-444b-878f-09b445e8c3a2 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.345439 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc15f28-0ab6-444b-878f-09b445e8c3a2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.349236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc15f28-0ab6-444b-878f-09b445e8c3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.369085 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-8dc15f28-0ab6-444b-878f-09b445e8c3a2 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:53.370284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8dc15f28-0ab6-444b-878f-09b445e8c3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:53.376869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a42a7-fa32-400b-973a-a32779c46a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:53.378022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a42a7-fa32-400b-973a-a32779c46a42 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.378506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a42a7-fa32-400b-973a-a32779c46a42 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.379548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a42a7-fa32-400b-973a-a32779c46a42 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.390449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a42a7-fa32-400b-973a-a32779c46a42 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:53.392192 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a42a7-fa32-400b-973a-a32779c46a42 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-0e1a42a7-fa32-400b-973a-a32779c46a42 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:53.413947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:53.414969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:53.415948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:53.415948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:53.415948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:53.415948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:53.415948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:53.415948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:53.415948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:53.415948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:53.415948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:40:53.415948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:53.418646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 313/624] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:40:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:53.419722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a42a7-fa32-400b-973a-a32779c46a42 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:53.432079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:53.446695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:53.460425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 312/625] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:53.465280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc15f28-0ab6-444b-878f-09b445e8c3a2 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:53.566057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a42a7-fa32-400b-973a-a32779c46a42 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:53.600270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc15f28-0ab6-444b-878f-09b445e8c3a2 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:53.618891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a859732-5222-400f-b93f-027fb91d8ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:53.619702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a859732-5222-400f-b93f-027fb91d8ac6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.619981 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a859732-5222-400f-b93f-027fb91d8ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.620254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a859732-5222-400f-b93f-027fb91d8ac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.636303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a859732-5222-400f-b93f-027fb91d8ac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:53.636952 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a859732-5222-400f-b93f-027fb91d8ac6 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:53.653457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01310a82-4054-4861-9312-d7707f680d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:53.655130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01310a82-4054-4861-9312-d7707f680d3e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.656738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01310a82-4054-4861-9312-d7707f680d3e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-2a859732-5222-400f-b93f-027fb91d8ac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:53.656941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:53.657817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:53.658757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:53.658757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:53.658757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:53.658757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:53.659467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01310a82-4054-4861-9312-d7707f680d3e None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.659467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01310a82-4054-4861-9312-d7707f680d3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:53.659467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01310a82-4054-4861-9312-d7707f680d3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:53.660679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 314/626] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:53.661845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01310a82-4054-4861-9312-d7707f680d3e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:53.662585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638b3a-2faf-49d4-abb6-357ceac9bc6a None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.662885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638b3a-2faf-49d4-abb6-357ceac9bc6a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.663165 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638b3a-2faf-49d4-abb6-357ceac9bc6a None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.664666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 315/627] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:53.665297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48638b3a-2faf-49d4-abb6-357ceac9bc6a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:53.688621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 313/628] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jul 19 20:40:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:53.689192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01310a82-4054-4861-9312-d7707f680d3e None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:53.750776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:53.770654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:53.827667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48638b3a-2faf-49d4-abb6-357ceac9bc6a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:53.828670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01310a82-4054-4861-9312-d7707f680d3e None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:53.865033 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce722f-f138-4027-b755-e83e9c5b2ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:53.865788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce722f-f138-4027-b755-e83e9c5b2ca2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.866068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce722f-f138-4027-b755-e83e9c5b2ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.866189 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfeb1d-9b58-4226-9451-cab5c6044aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:53.866383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce722f-f138-4027-b755-e83e9c5b2ca2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.866804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfeb1d-9b58-4226-9451-cab5c6044aee None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.867084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfeb1d-9b58-4226-9451-cab5c6044aee None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.867359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfeb1d-9b58-4226-9451-cab5c6044aee None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.868081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfeb1d-9b58-4226-9451-cab5c6044aee None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:53.868421 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfeb1d-9b58-4226-9451-cab5c6044aee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:53.869851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfeb1d-9b58-4226-9451-cab5c6044aee None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:53.882106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce722f-f138-4027-b755-e83e9c5b2ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:53.886772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 314/629] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:53.887903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cbc50-ae14-4d2d-8e3e-e4f59cc4243b None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:53.888194 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cbc50-ae14-4d2d-8e3e-e4f59cc4243b None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:53.888501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cbc50-ae14-4d2d-8e3e-e4f59cc4243b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-51ce722f-f138-4027-b755-e83e9c5b2ca2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:53.890390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:53.891353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:53.892205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 316/630] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:53.893547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce722f-f138-4027-b755-e83e9c5b2ca2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:53.914093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-287cbc50-ae14-4d2d-8e3e-e4f59cc4243b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:53.915284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-287cbc50-ae14-4d2d-8e3e-e4f59cc4243b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:53.983277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 315/631] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:53.984039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cbc50-ae14-4d2d-8e3e-e4f59cc4243b None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:54.005804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce722f-f138-4027-b755-e83e9c5b2ca2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.011546 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:54.035011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:54.040672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097615b5-2174-4213-bc4b-2ee2405ca1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.041609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097615b5-2174-4213-bc4b-2ee2405ca1db None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.041887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097615b5-2174-4213-bc4b-2ee2405ca1db None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.042162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097615b5-2174-4213-bc4b-2ee2405ca1db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.052715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097615b5-2174-4213-bc4b-2ee2405ca1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:54.054158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097615b5-2174-4213-bc4b-2ee2405ca1db None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-097615b5-2174-4213-bc4b-2ee2405ca1db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:54.071541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:54.072595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:54.073711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:54.073711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:54.073711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:54.073711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:54.073711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 317/632] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:54.073711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097615b5-2174-4213-bc4b-2ee2405ca1db None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:54.087139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cbc50-ae14-4d2d-8e3e-e4f59cc4243b None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.120816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512d5e0-42e8-4d47-b6df-7e3a2b5e6ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.121627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512d5e0-42e8-4d47-b6df-7e3a2b5e6ac0 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.121914 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512d5e0-42e8-4d47-b6df-7e3a2b5e6ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.122192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512d5e0-42e8-4d47-b6df-7e3a2b5e6ac0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.130329 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512d5e0-42e8-4d47-b6df-7e3a2b5e6ac0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:54.131794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512d5e0-42e8-4d47-b6df-7e3a2b5e6ac0 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-3512d5e0-42e8-4d47-b6df-7e3a2b5e6ac0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:54.146826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:54.147819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:54.149508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:54.149508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:54.149508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:54.149508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:54.149508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:54.149508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:54.149508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:54.149508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:54.149508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 316/633] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:54.149508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ca7fd-4a61-4e43-8f88-4bddc004b597 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.150194 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ca7fd-4a61-4e43-8f88-4bddc004b597 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.150194 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ca7fd-4a61-4e43-8f88-4bddc004b597 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.151172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 317/634] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:54.151900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ca7fd-4a61-4e43-8f88-4bddc004b597 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:54.185875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097615b5-2174-4213-bc4b-2ee2405ca1db None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.222782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ee02d-feeb-4f94-9b10-27d1040aa53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.223522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ee02d-feeb-4f94-9b10-27d1040aa53a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.223807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ee02d-feeb-4f94-9b10-27d1040aa53a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.224086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ee02d-feeb-4f94-9b10-27d1040aa53a None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.224861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ee02d-feeb-4f94-9b10-27d1040aa53a None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:54.225168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ee02d-feeb-4f94-9b10-27d1040aa53a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:54.226650 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ee02d-feeb-4f94-9b10-27d1040aa53a None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:54.234985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 318/635] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:54.235728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ee02d-feeb-4f94-9b10-27d1040aa53a None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:54.246955 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:54.260754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ca7fd-4a61-4e43-8f88-4bddc004b597 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.283658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:54.294186 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4f30a-612f-42b8-9c1b-b4b27e96c3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.294979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4f30a-612f-42b8-9c1b-b4b27e96c3fa None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.295259 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4f30a-612f-42b8-9c1b-b4b27e96c3fa None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.295533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4f30a-612f-42b8-9c1b-b4b27e96c3fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.305976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4f30a-612f-42b8-9c1b-b4b27e96c3fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-9dd4f30a-612f-42b8-9c1b-b4b27e96c3fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:54.313906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:54.315006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:54.315859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 318/636] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:54.316593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4f30a-612f-42b8-9c1b-b4b27e96c3fa None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:54.339191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ee02d-feeb-4f94-9b10-27d1040aa53a None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.373683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b40d9-399e-4291-bf22-119de7ea977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.374427 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b40d9-399e-4291-bf22-119de7ea977a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.374620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b40d9-399e-4291-bf22-119de7ea977a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.374896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b40d9-399e-4291-bf22-119de7ea977a None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.375609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b40d9-399e-4291-bf22-119de7ea977a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:54.375917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b40d9-399e-4291-bf22-119de7ea977a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:54.377324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b40d9-399e-4291-bf22-119de7ea977a None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:54.389573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 319/637] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:54.390863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32d1e-53fa-4310-a117-57f7a50569f2 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.391152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32d1e-53fa-4310-a117-57f7a50569f2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.391433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b32d1e-53fa-4310-a117-57f7a50569f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.407229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-00b32d1e-53fa-4310-a117-57f7a50569f2 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:54.408456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-00b32d1e-53fa-4310-a117-57f7a50569f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:54.418232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4f30a-612f-42b8-9c1b-b4b27e96c3fa None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.451619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d1040-50ab-4057-8f27-47be9077866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.452278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d1040-50ab-4057-8f27-47be9077866c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.452607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d1040-50ab-4057-8f27-47be9077866c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.452884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d1040-50ab-4057-8f27-47be9077866c None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.453663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d1040-50ab-4057-8f27-47be9077866c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:54.453968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d1040-50ab-4057-8f27-47be9077866c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:54.455380 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d1040-50ab-4057-8f27-47be9077866c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:54.467223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 319/638] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:54.468016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d1040-50ab-4057-8f27-47be9077866c None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:54.479574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 320/639] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:54.479837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:54.480442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b32d1e-53fa-4310-a117-57f7a50569f2 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:54.530136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:54.578041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d1040-50ab-4057-8f27-47be9077866c None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.595979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b32d1e-53fa-4310-a117-57f7a50569f2 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.612707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c940d3-d402-432d-99e1-c830a220b4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.613502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c940d3-d402-432d-99e1-c830a220b4b9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.613842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c940d3-d402-432d-99e1-c830a220b4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.614618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c940d3-d402-432d-99e1-c830a220b4b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.624824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c940d3-d402-432d-99e1-c830a220b4b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:54.626258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c940d3-d402-432d-99e1-c830a220b4b9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:54.632146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48aa903-baf6-4c89-ae6c-bf292960e5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.632844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aa903-baf6-4c89-ae6c-bf292960e5f9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.633127 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aa903-baf6-4c89-ae6c-bf292960e5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.633397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aa903-baf6-4c89-ae6c-bf292960e5f9 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.634129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48aa903-baf6-4c89-ae6c-bf292960e5f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:54.634433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48aa903-baf6-4c89-ae6c-bf292960e5f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:54.635809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48aa903-baf6-4c89-ae6c-bf292960e5f9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:54.643705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 321/640] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-e6c940d3-d402-432d-99e1-c830a220b4b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:54.643895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:54.644791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:54.645736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:54.645736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:54.645736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:54.645736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:54.645960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48aa903-baf6-4c89-ae6c-bf292960e5f9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:54.646039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 320/641] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:54.646039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c940d3-d402-432d-99e1-c830a220b4b9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:54.705540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:54.740465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:54.745932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c940d3-d402-432d-99e1-c830a220b4b9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.746902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48aa903-baf6-4c89-ae6c-bf292960e5f9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.778226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761b9f2-ad9e-48bd-a364-c743f70e3e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.778896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761b9f2-ad9e-48bd-a364-c743f70e3e85 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.779143 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761b9f2-ad9e-48bd-a364-c743f70e3e85 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.779414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761b9f2-ad9e-48bd-a364-c743f70e3e85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.780284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761b9f2-ad9e-48bd-a364-c743f70e3e85 None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=928b8b2b510b4c939b37b2c0dfa57060) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:54.780416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c8a5d-11fa-4f9b-b65c-4c5bb11c7b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.780594 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761b9f2-ad9e-48bd-a364-c743f70e3e85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:54.781188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c8a5d-11fa-4f9b-b65c-4c5bb11c7b00 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.781466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c8a5d-11fa-4f9b-b65c-4c5bb11c7b00 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.781737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c8a5d-11fa-4f9b-b65c-4c5bb11c7b00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.782011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761b9f2-ad9e-48bd-a364-c743f70e3e85 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:54.788775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 321/642] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:54 2022] GET /identity/v3/role_assignments?user.id=928b8b2b510b4c939b37b2c0dfa57060&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:54.789498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761b9f2-ad9e-48bd-a364-c743f70e3e85 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:54.803584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c8a5d-11fa-4f9b-b65c-4c5bb11c7b00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:54.805142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c8a5d-11fa-4f9b-b65c-4c5bb11c7b00 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-785c8a5d-11fa-4f9b-b65c-4c5bb11c7b00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:54.825062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:54.826097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:54.832521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:54.832521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:54.832521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:54.832521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:54.832521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:54.832521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:54.832521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:54.832521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:54.832521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 322/643] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:54.832521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c8a5d-11fa-4f9b-b65c-4c5bb11c7b00 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:54.938418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761b9f2-ad9e-48bd-a364-c743f70e3e85 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.941698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c8a5d-11fa-4f9b-b65c-4c5bb11c7b00 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:54.968037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:54.981611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c7dce-d885-46dc-b7a7-3ef35e99a294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.982456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c7dce-d885-46dc-b7a7-3ef35e99a294 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.982579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710da9c-f8f3-44cb-893b-00b141f38a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:54.982774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c7dce-d885-46dc-b7a7-3ef35e99a294 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.983058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c7dce-d885-46dc-b7a7-3ef35e99a294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.983567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710da9c-f8f3-44cb-893b-00b141f38a19 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:54.983855 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710da9c-f8f3-44cb-893b-00b141f38a19 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:54.984129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710da9c-f8f3-44cb-893b-00b141f38a19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:54.995732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c7dce-d885-46dc-b7a7-3ef35e99a294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:54.997938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c7dce-d885-46dc-b7a7-3ef35e99a294 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:54.998074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710da9c-f8f3-44cb-893b-00b141f38a19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-a710da9c-f8f3-44cb-893b-00b141f38a19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:55.006218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:55.007890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 322/644] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:55.009376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710da9c-f8f3-44cb-893b-00b141f38a19 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-257c7dce-d885-46dc-b7a7-3ef35e99a294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:55.031577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:55.034285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:55.037424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:55.037424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:55.037424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:55.037424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:55.037424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 323/645] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:55.037424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c7dce-d885-46dc-b7a7-3ef35e99a294 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:55.057285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:55.129657 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710da9c-f8f3-44cb-893b-00b141f38a19 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:55.163683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c7dce-d885-46dc-b7a7-3ef35e99a294 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:55.166799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13555fa1-5ada-4495-9190-287a8cfdd830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:55.167474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13555fa1-5ada-4495-9190-287a8cfdd830 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:55.167765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13555fa1-5ada-4495-9190-287a8cfdd830 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:55.168044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13555fa1-5ada-4495-9190-287a8cfdd830 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:55.168814 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13555fa1-5ada-4495-9190-287a8cfdd830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:55.169124 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13555fa1-5ada-4495-9190-287a8cfdd830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:55.170497 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13555fa1-5ada-4495-9190-287a8cfdd830 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:55.184392 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 323/646] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:55.185641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13555fa1-5ada-4495-9190-287a8cfdd830 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:55.206678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcf12d-8525-468a-b280-191ba6be6040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:55.207969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcf12d-8525-468a-b280-191ba6be6040 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:55.208260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcf12d-8525-468a-b280-191ba6be6040 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:55.208576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcf12d-8525-468a-b280-191ba6be6040 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:55.209365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcf12d-8525-468a-b280-191ba6be6040 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:55.209673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcf12d-8525-468a-b280-191ba6be6040 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:55.211218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dcf12d-8525-468a-b280-191ba6be6040 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:55.227125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 324/647] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:55.228729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcf12d-8525-468a-b280-191ba6be6040 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:55.254929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:55.311704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:55.321877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13555fa1-5ada-4495-9190-287a8cfdd830 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:55.373539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadf59c-5675-428f-bc64-5198f37ae7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:55.373784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcf12d-8525-468a-b280-191ba6be6040 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:55.374210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadf59c-5675-428f-bc64-5198f37ae7a0 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:55.374497 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadf59c-5675-428f-bc64-5198f37ae7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:55.374783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadf59c-5675-428f-bc64-5198f37ae7a0 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:55.375538 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dadf59c-5675-428f-bc64-5198f37ae7a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:55.375850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dadf59c-5675-428f-bc64-5198f37ae7a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:55.377246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dadf59c-5675-428f-bc64-5198f37ae7a0 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:55.391125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 324/648] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:55.391879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadf59c-5675-428f-bc64-5198f37ae7a0 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:55.416672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32371193-d7a5-40e9-a506-440424e185ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:55.417361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32371193-d7a5-40e9-a506-440424e185ba None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:55.417760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32371193-d7a5-40e9-a506-440424e185ba None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:55.418230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32371193-d7a5-40e9-a506-440424e185ba None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:55.419482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32371193-d7a5-40e9-a506-440424e185ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:55.419944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32371193-d7a5-40e9-a506-440424e185ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:55.421373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32371193-d7a5-40e9-a506-440424e185ba None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:55.435510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 325/649] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:55.436776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32371193-d7a5-40e9-a506-440424e185ba None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:55.518396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadf59c-5675-428f-bc64-5198f37ae7a0 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:55.531144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:55.559469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32371193-d7a5-40e9-a506-440424e185ba None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:55.564911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4800a-5fa8-47d4-a09a-153d7408695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:55.565924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4800a-5fa8-47d4-a09a-153d7408695d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:55.566395 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4800a-5fa8-47d4-a09a-153d7408695d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:55.566879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4800a-5fa8-47d4-a09a-153d7408695d None admin] PATH_INFO: `/v3/users/glance` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:55.578255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4800a-5fa8-47d4-a09a-153d7408695d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:55.580755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4800a-5fa8-47d4-a09a-153d7408695d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:55.593474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-68b4800a-5fa8-47d4-a09a-153d7408695d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:55.596609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:55.597571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:55.599047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:55.599047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:55.599047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:55.599047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:55.599047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:55.599047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:55.599047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:55.599047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:55.599047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 20:40:55.599047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:55.599047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 325/650] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:55.599673 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4800a-5fa8-47d4-a09a-153d7408695d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:55.608242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ab309-a942-4641-84d3-1c491a4840af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:55.609507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ab309-a942-4641-84d3-1c491a4840af None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:55.609507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ab309-a942-4641-84d3-1c491a4840af None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:55.609800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ab309-a942-4641-84d3-1c491a4840af None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/812342d822f54aaf98e324d5d4e56753/roles/b2e0a9d2d13844e980a69f441f36d492` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:55.635241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ab309-a942-4641-84d3-1c491a4840af None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=812342d822f54aaf98e324d5d4e56753, role_id=b2e0a9d2d13844e980a69f441f36d492)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:55.637581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ab309-a942-4641-84d3-1c491a4840af None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:55.660086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 326/651] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:40:55 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/812342d822f54aaf98e324d5d4e56753/roles/b2e0a9d2d13844e980a69f441f36d492 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:40:55.661869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ab309-a942-4641-84d3-1c491a4840af None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:55.772309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4800a-5fa8-47d4-a09a-153d7408695d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:55.793252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ab309-a942-4641-84d3-1c491a4840af None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:55.815045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff7c64-b159-4406-abda-96a93392eb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:55.816153 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff7c64-b159-4406-abda-96a93392eb30 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:55.816778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff7c64-b159-4406-abda-96a93392eb30 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:55.817270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff7c64-b159-4406-abda-96a93392eb30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:55.830388 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff7c64-b159-4406-abda-96a93392eb30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:55.831887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cade6-40f6-43f3-8d49-da7c6154953c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:55.833383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff7c64-b159-4406-abda-96a93392eb30 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:55.833520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cade6-40f6-43f3-8d49-da7c6154953c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:55.833679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cade6-40f6-43f3-8d49-da7c6154953c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:55.833941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cade6-40f6-43f3-8d49-da7c6154953c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:55.844034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cade6-40f6-43f3-8d49-da7c6154953c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:55.845690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cade6-40f6-43f3-8d49-da7c6154953c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-baff7c64-b159-4406-abda-96a93392eb30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:55.858543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:55.859771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:55.860864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:55.860864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:55.860864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:55.860864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-301cade6-40f6-43f3-8d49-da7c6154953c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:55.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:55.862855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:55.864185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:55.864185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:55.864185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:55.864185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:55.864185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:55.864185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:55.864185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:55.864185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:55.864185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:40:55.864185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:55.864185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 327/653] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:55.864884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 326/653] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:55.864960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cade6-40f6-43f3-8d49-da7c6154953c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:55.873960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff7c64-b159-4406-abda-96a93392eb30 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:55.899777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:55.914581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:55.979214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cade6-40f6-43f3-8d49-da7c6154953c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:56.011459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff7c64-b159-4406-abda-96a93392eb30 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:56.020788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f905fc8-c6d5-4c9f-a532-640355dcda0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:56.021463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f905fc8-c6d5-4c9f-a532-640355dcda0a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:56.021748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f905fc8-c6d5-4c9f-a532-640355dcda0a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:56.022026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f905fc8-c6d5-4c9f-a532-640355dcda0a None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:56.022862 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f905fc8-c6d5-4c9f-a532-640355dcda0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:56.023171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f905fc8-c6d5-4c9f-a532-640355dcda0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:56.024610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f905fc8-c6d5-4c9f-a532-640355dcda0a None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:56.043264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 328/654] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jul 19 20:40:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:56.044153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f905fc8-c6d5-4c9f-a532-640355dcda0a None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:56.047738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b032b-290f-4d0b-b679-fa98b2f72c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:56.048676 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b032b-290f-4d0b-b679-fa98b2f72c57 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:56.048976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b032b-290f-4d0b-b679-fa98b2f72c57 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:56.049257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b032b-290f-4d0b-b679-fa98b2f72c57 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:56.049993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b032b-290f-4d0b-b679-fa98b2f72c57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:56.050304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b032b-290f-4d0b-b679-fa98b2f72c57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:56.051692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b032b-290f-4d0b-b679-fa98b2f72c57 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:56.065002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 327/655] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:56.065676 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b032b-290f-4d0b-b679-fa98b2f72c57 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:56.147020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:56.155023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f905fc8-c6d5-4c9f-a532-640355dcda0a None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:56.159053 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:56.183649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b032b-290f-4d0b-b679-fa98b2f72c57 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:56.194561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7e963-c351-4d45-959f-eed9dab19aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:56.195517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7e963-c351-4d45-959f-eed9dab19aa3 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:56.195948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7e963-c351-4d45-959f-eed9dab19aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:56.196445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7e963-c351-4d45-959f-eed9dab19aa3 None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:56.197472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7e963-c351-4d45-959f-eed9dab19aa3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:56.197943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7e963-c351-4d45-959f-eed9dab19aa3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:56.199824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7e963-c351-4d45-959f-eed9dab19aa3 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:56.231055 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585c60f-11fb-4b78-80e6-934fc7e52921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:56.231738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c60f-11fb-4b78-80e6-934fc7e52921 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:56.232027 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c60f-11fb-4b78-80e6-934fc7e52921 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:56.232309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c60f-11fb-4b78-80e6-934fc7e52921 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:56.233969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 329/656] 10.222.0.85 () {56 vars in 1137 bytes} [Tue Jul 19 20:40:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:56.235378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7e963-c351-4d45-959f-eed9dab19aa3 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:56.307511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585c60f-11fb-4b78-80e6-934fc7e52921 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:56.309065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585c60f-11fb-4b78-80e6-934fc7e52921 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:56.347418 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7e963-c351-4d45-959f-eed9dab19aa3 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:56.387181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee941dbd-5e7c-44d8-9a3c-c2ecafc34136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:56.388417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee941dbd-5e7c-44d8-9a3c-c2ecafc34136 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:56.388906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee941dbd-5e7c-44d8-9a3c-c2ecafc34136 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:56.389397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee941dbd-5e7c-44d8-9a3c-c2ecafc34136 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:56.392933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:56.398491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee941dbd-5e7c-44d8-9a3c-c2ecafc34136 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:56.400207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee941dbd-5e7c-44d8-9a3c-c2ecafc34136 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-ee941dbd-5e7c-44d8-9a3c-c2ecafc34136 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:56.415588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:56.416600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:56.417542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:56.417542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:56.417542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:56.417542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:56.417542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:56.417542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:56.417542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:56.417542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:56.417542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 20:40:56.417542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:56.420245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 330/657] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:56.421241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee941dbd-5e7c-44d8-9a3c-c2ecafc34136 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:56.434708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 328/658] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:56.436375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585c60f-11fb-4b78-80e6-934fc7e52921 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:56.438742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:56.541855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee941dbd-5e7c-44d8-9a3c-c2ecafc34136 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:56.586775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfcddd-4d04-4c67-8290-78f2b08fa72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:56.587475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfcddd-4d04-4c67-8290-78f2b08fa72c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:56.587761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfcddd-4d04-4c67-8290-78f2b08fa72c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:56.588038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfcddd-4d04-4c67-8290-78f2b08fa72c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:56.598774 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585c60f-11fb-4b78-80e6-934fc7e52921 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:56.653442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa26fdc-311b-46e2-a953-4ef422f8f612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:56.654438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa26fdc-311b-46e2-a953-4ef422f8f612 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:56.654900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa26fdc-311b-46e2-a953-4ef422f8f612 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:56.655342 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa26fdc-311b-46e2-a953-4ef422f8f612 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:56.656770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa26fdc-311b-46e2-a953-4ef422f8f612 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:56.657283 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa26fdc-311b-46e2-a953-4ef422f8f612 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:56.659704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa26fdc-311b-46e2-a953-4ef422f8f612 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:56.665071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfcddd-4d04-4c67-8290-78f2b08fa72c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:56.666604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfcddd-4d04-4c67-8290-78f2b08fa72c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:56.683153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:56.696747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 329/659] 10.222.0.85 () {56 vars in 1139 bytes} [Tue Jul 19 20:40:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:56.698162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa26fdc-311b-46e2-a953-4ef422f8f612 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:56.755319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:56.784261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 331/660] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:56.790227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfcddd-4d04-4c67-8290-78f2b08fa72c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:56.872422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa26fdc-311b-46e2-a953-4ef422f8f612 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:56.905647 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfcddd-4d04-4c67-8290-78f2b08fa72c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:56.929628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8cd57-cca7-4690-afa4-f2a9ee288ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:56.930802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8cd57-cca7-4690-afa4-f2a9ee288ba6 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:56.931240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8cd57-cca7-4690-afa4-f2a9ee288ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:56.931891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8cd57-cca7-4690-afa4-f2a9ee288ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:56.934880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:56.943889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8cd57-cca7-4690-afa4-f2a9ee288ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:56.945395 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8cd57-cca7-4690-afa4-f2a9ee288ba6 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-50d8cd57-cca7-4690-afa4-f2a9ee288ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:56.964854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:56.966694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:56.968699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:56.968699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:56.968699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:56.968699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:56.970821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 330/661] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:56.971848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8cd57-cca7-4690-afa4-f2a9ee288ba6 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:56.980658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a0d45-af23-4259-a58e-cd2c5701b42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:56.981617 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a0d45-af23-4259-a58e-cd2c5701b42f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:56.982275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a0d45-af23-4259-a58e-cd2c5701b42f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:56.982725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a0d45-af23-4259-a58e-cd2c5701b42f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:57.063572 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a0d45-af23-4259-a58e-cd2c5701b42f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:57.065080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a0d45-af23-4259-a58e-cd2c5701b42f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:57.087089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:57.106220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8cd57-cca7-4690-afa4-f2a9ee288ba6 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:57.143482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c79bf-f5e7-493c-8c8e-7bf7793d583c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:57.144274 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c79bf-f5e7-493c-8c8e-7bf7793d583c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:57.144578 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c79bf-f5e7-493c-8c8e-7bf7793d583c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:57.145383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c79bf-f5e7-493c-8c8e-7bf7793d583c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:57.157589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c79bf-f5e7-493c-8c8e-7bf7793d583c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:57.159069 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c79bf-f5e7-493c-8c8e-7bf7793d583c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-446c79bf-f5e7-493c-8c8e-7bf7793d583c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:57.179336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:57.180561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:57.181633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:57.181633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:57.181633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:57.181633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:57.185876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 331/662] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:57.186862 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c79bf-f5e7-493c-8c8e-7bf7793d583c None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:57.195914 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 332/663] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:40:57.197181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a0d45-af23-4259-a58e-cd2c5701b42f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:57.218505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:57.313805 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c79bf-f5e7-493c-8c8e-7bf7793d583c None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:57.326341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a0d45-af23-4259-a58e-cd2c5701b42f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:57.352607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728eecd-5749-4b89-9466-6260f426a968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:57.353284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728eecd-5749-4b89-9466-6260f426a968 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:57.353572 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728eecd-5749-4b89-9466-6260f426a968 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:57.353869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728eecd-5749-4b89-9466-6260f426a968 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:57.354615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728eecd-5749-4b89-9466-6260f426a968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:57.354928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728eecd-5749-4b89-9466-6260f426a968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:57.360384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728eecd-5749-4b89-9466-6260f426a968 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:57.366128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:57.372758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 332/664] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:57.374461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728eecd-5749-4b89-9466-6260f426a968 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:57.380000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10ff86-5d87-48d4-bdb1-99e771a2f9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:57.380725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10ff86-5d87-48d4-bdb1-99e771a2f9a0 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:57.381013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10ff86-5d87-48d4-bdb1-99e771a2f9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:57.381289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10ff86-5d87-48d4-bdb1-99e771a2f9a0 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:57.382017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10ff86-5d87-48d4-bdb1-99e771a2f9a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:57.382327 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10ff86-5d87-48d4-bdb1-99e771a2f9a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:57.383716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10ff86-5d87-48d4-bdb1-99e771a2f9a0 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:57.398408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 333/665] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:57.399786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f300d-b54a-4c8e-bd81-1f60f640c037 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:57.400083 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f300d-b54a-4c8e-bd81-1f60f640c037 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:57.400385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f300d-b54a-4c8e-bd81-1f60f640c037 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:57.404370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 334/666] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:57.404370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f300d-b54a-4c8e-bd81-1f60f640c037 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:57.492081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:57.500406 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728eecd-5749-4b89-9466-6260f426a968 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:57.521351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f300d-b54a-4c8e-bd81-1f60f640c037 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:57.547504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d3eeb-0416-45e7-9f13-afce96b8fb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:57.548834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d3eeb-0416-45e7-9f13-afce96b8fb4b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:57.549462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d3eeb-0416-45e7-9f13-afce96b8fb4b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:57.549939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d3eeb-0416-45e7-9f13-afce96b8fb4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:57.565656 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d3eeb-0416-45e7-9f13-afce96b8fb4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:57.567864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d3eeb-0416-45e7-9f13-afce96b8fb4b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:57.568547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3767789-2d7b-4083-9c81-424b9b9d2a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:57.569506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3767789-2d7b-4083-9c81-424b9b9d2a74 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:57.569809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3767789-2d7b-4083-9c81-424b9b9d2a74 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:57.570295 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3767789-2d7b-4083-9c81-424b9b9d2a74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:57.579813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3767789-2d7b-4083-9c81-424b9b9d2a74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:57.581288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3767789-2d7b-4083-9c81-424b9b9d2a74 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-a93d3eeb-0416-45e7-9f13-afce96b8fb4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:57.587432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:57.588407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:57.594372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:57.594372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:57.594372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:57.594372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:57.594372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 333/667] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:57.594372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d3eeb-0416-45e7-9f13-afce96b8fb4b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-d3767789-2d7b-4083-9c81-424b9b9d2a74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:57.597556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:57.598507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:57.605429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:57.605429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:57.605429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:57.605429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:57.605429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:57.605429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:57.605429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:57.605429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:57.605429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 335/668] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:57.605429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107a349-0d63-41e5-8a87-87fc52cf6351 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:57.605429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107a349-0d63-41e5-8a87-87fc52cf6351 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:57.606162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107a349-0d63-41e5-8a87-87fc52cf6351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:57.623850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4107a349-0d63-41e5-8a87-87fc52cf6351 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:57.625700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4107a349-0d63-41e5-8a87-87fc52cf6351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:57.651977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:57.704355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 336/669] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:57.704796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107a349-0d63-41e5-8a87-87fc52cf6351 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:57.716775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d3eeb-0416-45e7-9f13-afce96b8fb4b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:57.766066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0c79b-f860-459a-b148-ac94886d37f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:57.766940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0c79b-f860-459a-b148-ac94886d37f2 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:57.767621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0c79b-f860-459a-b148-ac94886d37f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:57.767945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0c79b-f860-459a-b148-ac94886d37f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:57.778776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0c79b-f860-459a-b148-ac94886d37f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:57.780260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0c79b-f860-459a-b148-ac94886d37f2 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:57.785290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-1df0c79b-f860-459a-b148-ac94886d37f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:57.795860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:57.796845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:57.798384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:57.798384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:57.798384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:57.798384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:57.798384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:57.798384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:57.798384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:57.798384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:57.798384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 334/670] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:57.798384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0c79b-f860-459a-b148-ac94886d37f2 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:57.844037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107a349-0d63-41e5-8a87-87fc52cf6351 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:57.885493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db7b46-b750-494f-aa66-06c7c0db07e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:57.886535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db7b46-b750-494f-aa66-06c7c0db07e8 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:57.886820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db7b46-b750-494f-aa66-06c7c0db07e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:57.887091 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db7b46-b750-494f-aa66-06c7c0db07e8 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:57.887825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db7b46-b750-494f-aa66-06c7c0db07e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:57.888137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db7b46-b750-494f-aa66-06c7c0db07e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:57.889597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db7b46-b750-494f-aa66-06c7c0db07e8 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:57.903004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 337/671] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:57.903780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db7b46-b750-494f-aa66-06c7c0db07e8 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:57.929773 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0c79b-f860-459a-b148-ac94886d37f2 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:57.945422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:57.970850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac75ac-e719-4c08-a991-d80434c2ee66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:57.971506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac75ac-e719-4c08-a991-d80434c2ee66 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:57.971790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac75ac-e719-4c08-a991-d80434c2ee66 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:57.972057 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac75ac-e719-4c08-a991-d80434c2ee66 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:57.972888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac75ac-e719-4c08-a991-d80434c2ee66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:57.973222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac75ac-e719-4c08-a991-d80434c2ee66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:57.975501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac75ac-e719-4c08-a991-d80434c2ee66 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:57.987713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 335/672] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:57.989073 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac75ac-e719-4c08-a991-d80434c2ee66 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:58.034518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db7b46-b750-494f-aa66-06c7c0db07e8 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:58.072317 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:58.083883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750c669-be8d-40b8-aa3f-a6899fad7629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:58.084708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750c669-be8d-40b8-aa3f-a6899fad7629 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.084984 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750c669-be8d-40b8-aa3f-a6899fad7629 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.085256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750c669-be8d-40b8-aa3f-a6899fad7629 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.095390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a750c669-be8d-40b8-aa3f-a6899fad7629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-a750c669-be8d-40b8-aa3f-a6899fad7629 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:58.102959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:58.104114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 338/673] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:40:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:58.105012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750c669-be8d-40b8-aa3f-a6899fad7629 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:58.114892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac75ac-e719-4c08-a991-d80434c2ee66 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:58.149306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e504c1-6704-4e37-bd3a-8cea29a8ea3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:58.149962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e504c1-6704-4e37-bd3a-8cea29a8ea3f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.150247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e504c1-6704-4e37-bd3a-8cea29a8ea3f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.150534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e504c1-6704-4e37-bd3a-8cea29a8ea3f None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.151318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e504c1-6704-4e37-bd3a-8cea29a8ea3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:58.151629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e504c1-6704-4e37-bd3a-8cea29a8ea3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:58.153171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e504c1-6704-4e37-bd3a-8cea29a8ea3f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:58.164799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 336/674] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:58.166099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a50bf0-ce9d-4b10-bbc8-9266c1fca4c3 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.166391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a50bf0-ce9d-4b10-bbc8-9266c1fca4c3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.166671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a50bf0-ce9d-4b10-bbc8-9266c1fca4c3 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.168986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 337/675] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:40:58.168986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a50bf0-ce9d-4b10-bbc8-9266c1fca4c3 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:58.206555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:58.214470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750c669-be8d-40b8-aa3f-a6899fad7629 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:58.261653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8940e-f1ec-4d58-8c3f-a95c01d4fe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:58.262436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8940e-f1ec-4d58-8c3f-a95c01d4fe2d None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.262727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8940e-f1ec-4d58-8c3f-a95c01d4fe2d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.262998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8940e-f1ec-4d58-8c3f-a95c01d4fe2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.270192 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8940e-f1ec-4d58-8c3f-a95c01d4fe2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:58.271663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8940e-f1ec-4d58-8c3f-a95c01d4fe2d None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:58.282072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a50bf0-ce9d-4b10-bbc8-9266c1fca4c3 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-36d8940e-f1ec-4d58-8c3f-a95c01d4fe2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:40:58.286728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:58.287796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:58.288926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:40:58.288926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:40:58.288926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:40:58.288926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:40:58.288926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:40:58.288926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:40:58.288926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:40:58.288926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:58.292486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 339/676] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:40:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:58.293475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8940e-f1ec-4d58-8c3f-a95c01d4fe2d None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:58.326974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e23b-395b-447f-811e-d8820adb258e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:58.327627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e23b-395b-447f-811e-d8820adb258e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.327915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e23b-395b-447f-811e-d8820adb258e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.328195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e23b-395b-447f-811e-d8820adb258e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.329053 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3e23b-395b-447f-811e-d8820adb258e None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=e35e7720e3bf4c9e80bdefa6a1cb8296) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:58.329370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3e23b-395b-447f-811e-d8820adb258e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:58.330789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3e23b-395b-447f-811e-d8820adb258e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:58.336879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 338/677] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:58 2022] GET /identity/v3/role_assignments?user.id=e35e7720e3bf4c9e80bdefa6a1cb8296&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:58.337748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e23b-395b-447f-811e-d8820adb258e None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:58.351089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:58.445115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8940e-f1ec-4d58-8c3f-a95c01d4fe2d None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:58.451902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:58.464727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e23b-395b-447f-811e-d8820adb258e None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:58.493970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fd455-b3f8-4ab5-bfbb-bee9e5599b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:58.494627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fd455-b3f8-4ab5-bfbb-bee9e5599b02 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.494996 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fd455-b3f8-4ab5-bfbb-bee9e5599b02 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.495182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fd455-b3f8-4ab5-bfbb-bee9e5599b02 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.495950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fd455-b3f8-4ab5-bfbb-bee9e5599b02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:58.496263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fd455-b3f8-4ab5-bfbb-bee9e5599b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:58.497777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fd455-b3f8-4ab5-bfbb-bee9e5599b02 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:58.507164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cb35e-8fb1-423a-aa27-4379c1e82868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:58.507164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cb35e-8fb1-423a-aa27-4379c1e82868 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.507164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cb35e-8fb1-423a-aa27-4379c1e82868 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.507639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cb35e-8fb1-423a-aa27-4379c1e82868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.512517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 340/678] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:40:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:58.512688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280cb35e-8fb1-423a-aa27-4379c1e82868 None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=81411204bc9f4210880d807b41da9643) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:58.512688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280cb35e-8fb1-423a-aa27-4379c1e82868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:58.512869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0893984-e16f-4123-9b21-477336a062ce None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.512869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0893984-e16f-4123-9b21-477336a062ce None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.512869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0893984-e16f-4123-9b21-477336a062ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.513101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280cb35e-8fb1-423a-aa27-4379c1e82868 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:58.522116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 339/679] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:58 2022] GET /identity/v3/role_assignments?user.id=81411204bc9f4210880d807b41da9643&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:58.522116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cb35e-8fb1-423a-aa27-4379c1e82868 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:58.536293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c0893984-e16f-4123-9b21-477336a062ce None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:40:58.538436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c0893984-e16f-4123-9b21-477336a062ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:40:58.678322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 341/680] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:40:58.679300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0893984-e16f-4123-9b21-477336a062ce None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:58.719012 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cb35e-8fb1-423a-aa27-4379c1e82868 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:58.741057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:58.767933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9ce71-95e8-470e-883a-eac103aa508f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:58.769076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9ce71-95e8-470e-883a-eac103aa508f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.769553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9ce71-95e8-470e-883a-eac103aa508f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.770564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9ce71-95e8-470e-883a-eac103aa508f None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.771934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9ce71-95e8-470e-883a-eac103aa508f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:58.772425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9ce71-95e8-470e-883a-eac103aa508f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:58.774766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9ce71-95e8-470e-883a-eac103aa508f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:58.790672 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 340/681] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:40:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:58.791561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9ce71-95e8-470e-883a-eac103aa508f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:58.828752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:58.841511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0893984-e16f-4123-9b21-477336a062ce None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:58.893653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a1ab4-4199-4280-ae82-bd229c7af2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:58.895706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a1ab4-4199-4280-ae82-bd229c7af2ac None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:58.896367 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a1ab4-4199-4280-ae82-bd229c7af2ac None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:58.896877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a1ab4-4199-4280-ae82-bd229c7af2ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:58.922317 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a1ab4-4199-4280-ae82-bd229c7af2ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:58.926560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a1ab4-4199-4280-ae82-bd229c7af2ac None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-101a1ab4-4199-4280-ae82-bd229c7af2ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:58.959200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:58.960781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:58.962260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:58.962260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:58.962260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:58.962260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:58.962584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9ce71-95e8-470e-883a-eac103aa508f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:58.965545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 342/682] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:58.966791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a1ab4-4199-4280-ae82-bd229c7af2ac None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:59.010514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd550cb0-bcad-4d28-8d25-ef900cc91022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:59.011274 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550cb0-bcad-4d28-8d25-ef900cc91022 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:59.012000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550cb0-bcad-4d28-8d25-ef900cc91022 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:59.012378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550cb0-bcad-4d28-8d25-ef900cc91022 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:59.030924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd550cb0-bcad-4d28-8d25-ef900cc91022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-dd550cb0-bcad-4d28-8d25-ef900cc91022 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:40:59.040904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:59.041910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:40:59.042825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:59.048422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 341/683] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:40:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:59.049729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd550cb0-bcad-4d28-8d25-ef900cc91022 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:59.108783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:59.126636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a1ab4-4199-4280-ae82-bd229c7af2ac None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:59.164911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:59.178512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7cc8e-bf24-488a-a70e-56bbaf79efdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:59.179571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7cc8e-bf24-488a-a70e-56bbaf79efdf None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:59.180160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7cc8e-bf24-488a-a70e-56bbaf79efdf None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:59.180774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7cc8e-bf24-488a-a70e-56bbaf79efdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:59.182079 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7cc8e-bf24-488a-a70e-56bbaf79efdf None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=5db827b3a66843749e04376559aa27c2) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:59.182681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7cc8e-bf24-488a-a70e-56bbaf79efdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:59.184959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7cc8e-bf24-488a-a70e-56bbaf79efdf None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:59.187294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd550cb0-bcad-4d28-8d25-ef900cc91022 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:59.193334 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 343/684] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:40:58 2022] GET /identity/v3/role_assignments?user.id=5db827b3a66843749e04376559aa27c2&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:59.194475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7cc8e-bf24-488a-a70e-56bbaf79efdf None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:59.261937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d725173-de4d-4b9e-bb08-cf4572941abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:59.262614 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d725173-de4d-4b9e-bb08-cf4572941abb None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:59.262813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d725173-de4d-4b9e-bb08-cf4572941abb None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:59.263139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d725173-de4d-4b9e-bb08-cf4572941abb None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:59.264040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d725173-de4d-4b9e-bb08-cf4572941abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:59.264238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d725173-de4d-4b9e-bb08-cf4572941abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:59.265663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d725173-de4d-4b9e-bb08-cf4572941abb None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:59.288043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 342/685] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:59.294507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d725173-de4d-4b9e-bb08-cf4572941abb None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:59.350497 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7cc8e-bf24-488a-a70e-56bbaf79efdf None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:59.420616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d725173-de4d-4b9e-bb08-cf4572941abb None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:59.429096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aa59c-eb37-46f9-8c62-994de8253f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:59.430573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aa59c-eb37-46f9-8c62-994de8253f1d None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:59.431176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aa59c-eb37-46f9-8c62-994de8253f1d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:59.431968 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aa59c-eb37-46f9-8c62-994de8253f1d None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:59.433082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0aa59c-eb37-46f9-8c62-994de8253f1d None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:59.433616 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0aa59c-eb37-46f9-8c62-994de8253f1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:59.435657 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0aa59c-eb37-46f9-8c62-994de8253f1d None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:59.460899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd5048-b5ee-4f2a-a10e-a16e76654a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:59.461717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd5048-b5ee-4f2a-a10e-a16e76654a35 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:59.462004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd5048-b5ee-4f2a-a10e-a16e76654a35 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:59.462287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd5048-b5ee-4f2a-a10e-a16e76654a35 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:59.468318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 344/686] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:40:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:59.471219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd5048-b5ee-4f2a-a10e-a16e76654a35 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:59.472756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd5048-b5ee-4f2a-a10e-a16e76654a35 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:59.476697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:59.477420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aa59c-eb37-46f9-8c62-994de8253f1d None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-4ebd5048-b5ee-4f2a-a10e-a16e76654a35 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:40:59.491316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:59.492455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:59.494848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:59.494848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:59.494848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:40:59.494848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:40:59.494848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:40:59.494848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:40:59.494848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:40:59.494848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:40:59.494848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:40:59.494848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:40:59.494848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 343/687] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jul 19 20:40:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:40:59.502076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd5048-b5ee-4f2a-a10e-a16e76654a35 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:59.519061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:59.618221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd5048-b5ee-4f2a-a10e-a16e76654a35 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:59.623860 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aa59c-eb37-46f9-8c62-994de8253f1d None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:59.657850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a5f03-89e3-4d76-83fd-2bc523d4fdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:59.658825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a5f03-89e3-4d76-83fd-2bc523d4fdec None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:59.659198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a5f03-89e3-4d76-83fd-2bc523d4fdec None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:59.660239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a5f03-89e3-4d76-83fd-2bc523d4fdec None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:59.660747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a5f03-89e3-4d76-83fd-2bc523d4fdec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:59.661060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a5f03-89e3-4d76-83fd-2bc523d4fdec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:59.663009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a5f03-89e3-4d76-83fd-2bc523d4fdec None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:59.666888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53fcdb-bb76-456d-b94e-b1bbf9609759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:59.668798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fcdb-bb76-456d-b94e-b1bbf9609759 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:59.669567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fcdb-bb76-456d-b94e-b1bbf9609759 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:59.670262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fcdb-bb76-456d-b94e-b1bbf9609759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:59.686482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 344/688] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jul 19 20:40:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:40:59.687458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53fcdb-bb76-456d-b94e-b1bbf9609759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:59.689160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53fcdb-bb76-456d-b94e-b1bbf9609759 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:59.692763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a5f03-89e3-4d76-83fd-2bc523d4fdec None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-5c53fcdb-bb76-456d-b94e-b1bbf9609759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:40:59.718840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:40:59.719823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:40:59.720775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:40:59.720775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:40:59.720775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:40:59.720775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:40:59.720775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 345/689] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:40:59.726233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53fcdb-bb76-456d-b94e-b1bbf9609759 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:40:59.774985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:59.827423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:40:59.856742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a5f03-89e3-4d76-83fd-2bc523d4fdec None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:59.902028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53fcdb-bb76-456d-b94e-b1bbf9609759 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:40:59.932270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82399c4f-171d-45c5-880b-9bcb04644f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:59.933329 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82399c4f-171d-45c5-880b-9bcb04644f40 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:59.933823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82399c4f-171d-45c5-880b-9bcb04644f40 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:59.934323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82399c4f-171d-45c5-880b-9bcb04644f40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:59.948221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6e471-df96-4e55-b67b-6038e30ec830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:40:59.948996 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6e471-df96-4e55-b67b-6038e30ec830 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:40:59.949290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6e471-df96-4e55-b67b-6038e30ec830 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:40:59.949565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6e471-df96-4e55-b67b-6038e30ec830 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:40:59.950305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6e471-df96-4e55-b67b-6038e30ec830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:40:59.950677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6e471-df96-4e55-b67b-6038e30ec830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:40:59.952136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6e471-df96-4e55-b67b-6038e30ec830 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:40:59.973817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 346/690] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:40:59.979764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6e471-df96-4e55-b67b-6038e30ec830 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:00.098861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82399c4f-171d-45c5-880b-9bcb04644f40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:00.100729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82399c4f-171d-45c5-880b-9bcb04644f40 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:00.228563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:00.233436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6e471-df96-4e55-b67b-6038e30ec830 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:00.296924 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:00.313260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a869f6c-4712-48b8-b3d5-aefb4dd61754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:00.314433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a869f6c-4712-48b8-b3d5-aefb4dd61754 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:00.315555 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a869f6c-4712-48b8-b3d5-aefb4dd61754 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:00.316204 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a869f6c-4712-48b8-b3d5-aefb4dd61754 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:00.322918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 345/691] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:40:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 20:41:00.325882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82399c4f-171d-45c5-880b-9bcb04644f40 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:00.342398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a869f6c-4712-48b8-b3d5-aefb4dd61754 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:00.345731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a869f6c-4712-48b8-b3d5-aefb4dd61754 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-0a869f6c-4712-48b8-b3d5-aefb4dd61754 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:00.370898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:00.372057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:00.373411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:00.373411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:00.373411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:00.373411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:00.373411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:00.373411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:00.373411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:00.373411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:00.373411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:41:00.373411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:00.386651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 347/692] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:40:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:00.395209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a869f6c-4712-48b8-b3d5-aefb4dd61754 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:00.552846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82399c4f-171d-45c5-880b-9bcb04644f40 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:00.594728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a869f6c-4712-48b8-b3d5-aefb4dd61754 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:00.613134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3148e6c-030f-42ac-82b0-75a17e2c61ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:00.615000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3148e6c-030f-42ac-82b0-75a17e2c61ab None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:00.615545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3148e6c-030f-42ac-82b0-75a17e2c61ab None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:00.616030 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3148e6c-030f-42ac-82b0-75a17e2c61ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:00.639539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3148e6c-030f-42ac-82b0-75a17e2c61ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:00.641454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3148e6c-030f-42ac-82b0-75a17e2c61ab None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:00.654319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32055a15-4cea-4ece-88e8-834d211f8bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:00.662446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32055a15-4cea-4ece-88e8-834d211f8bdd None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:00.664509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32055a15-4cea-4ece-88e8-834d211f8bdd None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:00.665031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32055a15-4cea-4ece-88e8-834d211f8bdd None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:00.667008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32055a15-4cea-4ece-88e8-834d211f8bdd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:00.667731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32055a15-4cea-4ece-88e8-834d211f8bdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:00.669840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32055a15-4cea-4ece-88e8-834d211f8bdd None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-d3148e6c-030f-42ac-82b0-75a17e2c61ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:00.678447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:00.679413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:00.680351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:00.680351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:00.680351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:00.680351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:00.682937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 346/693] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:00.689861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3148e6c-030f-42ac-82b0-75a17e2c61ab None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:00.709393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 348/694] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jul 19 20:41:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:00.712285 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:00.713599 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32055a15-4cea-4ece-88e8-834d211f8bdd None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:00.797465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:00.881587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3148e6c-030f-42ac-82b0-75a17e2c61ab None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:00.905732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32055a15-4cea-4ece-88e8-834d211f8bdd None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:00.933586 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049530c-ee9e-41ce-8c48-b6868ef4afc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:00.934334 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049530c-ee9e-41ce-8c48-b6868ef4afc7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:00.934848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049530c-ee9e-41ce-8c48-b6868ef4afc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:00.935126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049530c-ee9e-41ce-8c48-b6868ef4afc7 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:00.936405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049530c-ee9e-41ce-8c48-b6868ef4afc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:00.937000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049530c-ee9e-41ce-8c48-b6868ef4afc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:00.938716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049530c-ee9e-41ce-8c48-b6868ef4afc7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:00.948354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6ebc-60c6-445e-895d-ee8202616cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:00.949327 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6ebc-60c6-445e-895d-ee8202616cea None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:00.949763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6ebc-60c6-445e-895d-ee8202616cea None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:00.950188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6ebc-60c6-445e-895d-ee8202616cea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:00.955262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 347/695] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:00.957043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dda876-4e02-4596-833d-500dc0cadf7f None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:00.958011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dda876-4e02-4596-833d-500dc0cadf7f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:00.958391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dda876-4e02-4596-833d-500dc0cadf7f None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:00.964707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 348/696] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:00.965829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dda876-4e02-4596-833d-500dc0cadf7f None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:01.073966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:01.075789 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6ebc-60c6-445e-895d-ee8202616cea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:01.079342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6ebc-60c6-445e-895d-ee8202616cea None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:01.118480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dda876-4e02-4596-833d-500dc0cadf7f None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:01.151894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:01.173079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58455709-894d-40d4-99e4-81f454da3132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:01.174001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58455709-894d-40d4-99e4-81f454da3132 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:01.174290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58455709-894d-40d4-99e4-81f454da3132 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:01.174572 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58455709-894d-40d4-99e4-81f454da3132 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:01.184400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58455709-894d-40d4-99e4-81f454da3132 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:01.186332 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58455709-894d-40d4-99e4-81f454da3132 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-58455709-894d-40d4-99e4-81f454da3132 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:01.205459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:01.207074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:01.209340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:01.209340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:01.209340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:01.209340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:01.209340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:01.209340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:01.209340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:01.209340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:01.209340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 349/697] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:01.209340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbe9d0-74a7-4a51-b724-30ca564de5c1 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:01.210350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbe9d0-74a7-4a51-b724-30ca564de5c1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:01.210350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbe9d0-74a7-4a51-b724-30ca564de5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:01.220539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 349/698] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 20:41:01.223903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7d208-f212-4d64-b8af-db8287a2121c None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:01.224468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7d208-f212-4d64-b8af-db8287a2121c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:01.224956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7d208-f212-4d64-b8af-db8287a2121c None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:01.231532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 350/699] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:01.231532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7d208-f212-4d64-b8af-db8287a2121c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:01.237329 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-6edbe9d0-74a7-4a51-b724-30ca564de5c1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:01.238577 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6edbe9d0-74a7-4a51-b724-30ca564de5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:01.352360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 350/700] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:41:01.352360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbe9d0-74a7-4a51-b724-30ca564de5c1 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:01.364790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7d208-f212-4d64-b8af-db8287a2121c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:01.403289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e35879-8fbb-4f0e-8979-4d0b4ed42b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:01.403951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e35879-8fbb-4f0e-8979-4d0b4ed42b5f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:01.404230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e35879-8fbb-4f0e-8979-4d0b4ed42b5f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:01.404534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e35879-8fbb-4f0e-8979-4d0b4ed42b5f None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:01.405311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e35879-8fbb-4f0e-8979-4d0b4ed42b5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:01.405618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e35879-8fbb-4f0e-8979-4d0b4ed42b5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:01.407035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e35879-8fbb-4f0e-8979-4d0b4ed42b5f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:01.419474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 351/701] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:01.420785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723f2a6-9e0b-4771-8c33-94a662354305 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:01.421077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723f2a6-9e0b-4771-8c33-94a662354305 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:01.421349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723f2a6-9e0b-4771-8c33-94a662354305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:01.425754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:01.428112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:01.440281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-a723f2a6-9e0b-4771-8c33-94a662354305 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:01.441534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a723f2a6-9e0b-4771-8c33-94a662354305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:01.496860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbe9d0-74a7-4a51-b724-30ca564de5c1 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:01.521800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 352/702] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:41:01.522732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723f2a6-9e0b-4771-8c33-94a662354305 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:01.543179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f112d-43b5-4eba-9fc1-673bfeb5cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:01.544383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f112d-43b5-4eba-9fc1-673bfeb5cc2e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:01.544908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f112d-43b5-4eba-9fc1-673bfeb5cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:01.545414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f112d-43b5-4eba-9fc1-673bfeb5cc2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:01.558451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f112d-43b5-4eba-9fc1-673bfeb5cc2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:01.559961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f112d-43b5-4eba-9fc1-673bfeb5cc2e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-b61f112d-43b5-4eba-9fc1-673bfeb5cc2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:01.582262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:01.583365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:01.584800 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:01.584800 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:01.584800 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:01.584800 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:01.585378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 351/703] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:01.586487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f112d-43b5-4eba-9fc1-673bfeb5cc2e None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:01.649386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723f2a6-9e0b-4771-8c33-94a662354305 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:01.687406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6f3d7-c5b6-48e7-a58b-db9d0633cad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:01.688148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6f3d7-c5b6-48e7-a58b-db9d0633cad6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:01.688462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6f3d7-c5b6-48e7-a58b-db9d0633cad6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:01.688748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6f3d7-c5b6-48e7-a58b-db9d0633cad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:01.690953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:01.702021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6f3d7-c5b6-48e7-a58b-db9d0633cad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:01.703857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6f3d7-c5b6-48e7-a58b-db9d0633cad6 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:01.728197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f112d-43b5-4eba-9fc1-673bfeb5cc2e None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-9db6f3d7-c5b6-48e7-a58b-db9d0633cad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:01.729775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:01.730810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:01.732037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:01.732037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:01.732037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:01.732037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:01.736156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 353/704] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:01.736871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6f3d7-c5b6-48e7-a58b-db9d0633cad6 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:01.744811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:01.769002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3934b28-9e8b-4308-9fa6-c5211027b78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:01.769665 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3934b28-9e8b-4308-9fa6-c5211027b78d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:01.770467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3934b28-9e8b-4308-9fa6-c5211027b78d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:01.770737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3934b28-9e8b-4308-9fa6-c5211027b78d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:01.771571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3934b28-9e8b-4308-9fa6-c5211027b78d None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=928b8b2b510b4c939b37b2c0dfa57060) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:01.771887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3934b28-9e8b-4308-9fa6-c5211027b78d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:01.773347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3934b28-9e8b-4308-9fa6-c5211027b78d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:01.780551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 352/705] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:01 2022] GET /identity/v3/role_assignments?user.id=928b8b2b510b4c939b37b2c0dfa57060&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:01.781229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3934b28-9e8b-4308-9fa6-c5211027b78d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:01.887539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6f3d7-c5b6-48e7-a58b-db9d0633cad6 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:01.918443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3934b28-9e8b-4308-9fa6-c5211027b78d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:01.931515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f827c85-65cc-4856-a6bd-7e1d514c69d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:01.932670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f827c85-65cc-4856-a6bd-7e1d514c69d5 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:01.932955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f827c85-65cc-4856-a6bd-7e1d514c69d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:01.933235 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f827c85-65cc-4856-a6bd-7e1d514c69d5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:01.933921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f827c85-65cc-4856-a6bd-7e1d514c69d5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:01.935356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f827c85-65cc-4856-a6bd-7e1d514c69d5 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-0f827c85-65cc-4856-a6bd-7e1d514c69d5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:01.950936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:01.952131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:41:01.953371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:01.956424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 354/706] 10.222.0.85 () {56 vars in 1129 bytes} [Tue Jul 19 20:41:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:01.958403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f827c85-65cc-4856-a6bd-7e1d514c69d5 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:01.971821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199e24b-54c0-4b2e-9422-8eb2848db7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:01.972561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199e24b-54c0-4b2e-9422-8eb2848db7a4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:01.972885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199e24b-54c0-4b2e-9422-8eb2848db7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:01.973686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199e24b-54c0-4b2e-9422-8eb2848db7a4 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:01.974422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199e24b-54c0-4b2e-9422-8eb2848db7a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:01.974733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199e24b-54c0-4b2e-9422-8eb2848db7a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:01.976362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199e24b-54c0-4b2e-9422-8eb2848db7a4 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:01.989892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 353/707] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:01.991535 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882b8f2-ae35-4e43-b2d5-ea640f6d3d58 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:01.991835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882b8f2-ae35-4e43-b2d5-ea640f6d3d58 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:01.992116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882b8f2-ae35-4e43-b2d5-ea640f6d3d58 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:01.996483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 354/708] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:01.996483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882b8f2-ae35-4e43-b2d5-ea640f6d3d58 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:02.020711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:02.040099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:02.089740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f827c85-65cc-4856-a6bd-7e1d514c69d5 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:02.119679 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882b8f2-ae35-4e43-b2d5-ea640f6d3d58 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:02.128352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58a612-5690-4948-b88b-593a0c6fa433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:02.129065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a612-5690-4948-b88b-593a0c6fa433 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:02.129859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a612-5690-4948-b88b-593a0c6fa433 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:02.130133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58a612-5690-4948-b88b-593a0c6fa433 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:02.130890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58a612-5690-4948-b88b-593a0c6fa433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:02.131201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58a612-5690-4948-b88b-593a0c6fa433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:02.132687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58a612-5690-4948-b88b-593a0c6fa433 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:02.155690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 355/709] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:02.156631 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58a612-5690-4948-b88b-593a0c6fa433 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:02.174068 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef4e5e-88d0-4f0b-84a0-ebfc049c3652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:02.174841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef4e5e-88d0-4f0b-84a0-ebfc049c3652 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:02.175135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef4e5e-88d0-4f0b-84a0-ebfc049c3652 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:02.175411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef4e5e-88d0-4f0b-84a0-ebfc049c3652 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:02.176157 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef4e5e-88d0-4f0b-84a0-ebfc049c3652 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:02.176517 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef4e5e-88d0-4f0b-84a0-ebfc049c3652 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:02.177890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef4e5e-88d0-4f0b-84a0-ebfc049c3652 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:02.187651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 355/710] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jul 19 20:41:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:02.187651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee6bd9-929c-4bed-ba4c-bbebef75fbce None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:02.187651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee6bd9-929c-4bed-ba4c-bbebef75fbce None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:02.187651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee6bd9-929c-4bed-ba4c-bbebef75fbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:02.222253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e9ee6bd9-929c-4bed-ba4c-bbebef75fbce None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:02.223684 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e9ee6bd9-929c-4bed-ba4c-bbebef75fbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:02.283391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58a612-5690-4948-b88b-593a0c6fa433 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:02.295696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:02.298795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 356/711] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:41:02.299595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee6bd9-929c-4bed-ba4c-bbebef75fbce None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:02.333140 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd6c0c-b943-4e8d-bf76-38d89205a819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:02.334338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6c0c-b943-4e8d-bf76-38d89205a819 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:02.334736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6c0c-b943-4e8d-bf76-38d89205a819 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:02.335076 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6c0c-b943-4e8d-bf76-38d89205a819 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:02.339804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:02.343558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd6c0c-b943-4e8d-bf76-38d89205a819 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:02.344933 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd6c0c-b943-4e8d-bf76-38d89205a819 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-f4bd6c0c-b943-4e8d-bf76-38d89205a819 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:02.361852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:02.362837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:02.363822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:02.363822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:02.363822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:02.363822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:02.363822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:02.363822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:02.363822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:02.363822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:02.363822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 356/712] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:02.364311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd6c0c-b943-4e8d-bf76-38d89205a819 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:02.445079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee6bd9-929c-4bed-ba4c-bbebef75fbce None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:02.481394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd6c0c-b943-4e8d-bf76-38d89205a819 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:02.491618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3eabc-23db-43ee-af3c-6f4b25bf70b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:02.492308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3eabc-23db-43ee-af3c-6f4b25bf70b8 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:02.492632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3eabc-23db-43ee-af3c-6f4b25bf70b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:02.492914 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3eabc-23db-43ee-af3c-6f4b25bf70b8 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:02.493633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3eabc-23db-43ee-af3c-6f4b25bf70b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:02.495280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3eabc-23db-43ee-af3c-6f4b25bf70b8 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:02.517729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 357/713] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:41:02 2022] POST /identity/v3/users => generated 254 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:02.517729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3eabc-23db-43ee-af3c-6f4b25bf70b8 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:02.534069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86792cb9-7bb6-42a6-8b3f-239e2491b303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:02.534855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86792cb9-7bb6-42a6-8b3f-239e2491b303 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:02.535207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86792cb9-7bb6-42a6-8b3f-239e2491b303 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:02.535586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86792cb9-7bb6-42a6-8b3f-239e2491b303 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:02.536440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86792cb9-7bb6-42a6-8b3f-239e2491b303 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:02.536828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86792cb9-7bb6-42a6-8b3f-239e2491b303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:02.538347 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86792cb9-7bb6-42a6-8b3f-239e2491b303 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:02.545669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 357/714] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jul 19 20:41:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:02.546645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86792cb9-7bb6-42a6-8b3f-239e2491b303 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:02.587343 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:02.714572 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:02.755435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3eabc-23db-43ee-af3c-6f4b25bf70b8 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:02.769025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86792cb9-7bb6-42a6-8b3f-239e2491b303 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:02.811655 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6c277-e9d4-48f0-b2ad-d4839f13b4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:02.812648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6c277-e9d4-48f0-b2ad-d4839f13b4f6 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:02.812932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6c277-e9d4-48f0-b2ad-d4839f13b4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:02.813205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6c277-e9d4-48f0-b2ad-d4839f13b4f6 None admin] PATH_INFO: `/v3/services/network` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:02.813870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6c277-e9d4-48f0-b2ad-d4839f13b4f6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:02.815243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6c277-e9d4-48f0-b2ad-d4839f13b4f6 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:02.816905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc83e14-6e03-45fc-ab6d-3a7df0f28af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:02.817759 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc83e14-6e03-45fc-ab6d-3a7df0f28af8 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:02.818113 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc83e14-6e03-45fc-ab6d-3a7df0f28af8 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:02.818397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc83e14-6e03-45fc-ab6d-3a7df0f28af8 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:02.819308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc83e14-6e03-45fc-ab6d-3a7df0f28af8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:02.819620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc83e14-6e03-45fc-ab6d-3a7df0f28af8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:02.821137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc83e14-6e03-45fc-ab6d-3a7df0f28af8 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-a7e6c277-e9d4-48f0-b2ad-d4839f13b4f6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:02.830620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:02.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:41:02.832799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:02.833564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 358/715] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:02 2022] GET /identity/v3/services/network => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:02.840264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6c277-e9d4-48f0-b2ad-d4839f13b4f6 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:02.844654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 358/716] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:02.857960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc83e14-6e03-45fc-ab6d-3a7df0f28af8 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:03.054902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6c277-e9d4-48f0-b2ad-d4839f13b4f6 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:03.064200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:03.088459 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc83e14-6e03-45fc-ab6d-3a7df0f28af8 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:03.149888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee0029-a6b3-4fec-bafa-bb7662fa1141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:03.150653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee0029-a6b3-4fec-bafa-bb7662fa1141 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:03.150944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee0029-a6b3-4fec-bafa-bb7662fa1141 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:03.151225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee0029-a6b3-4fec-bafa-bb7662fa1141 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:03.151943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee0029-a6b3-4fec-bafa-bb7662fa1141 None admin] RBAC: Adding query filter params (name=network) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:03.152253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee0029-a6b3-4fec-bafa-bb7662fa1141 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:03.153676 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee0029-a6b3-4fec-bafa-bb7662fa1141 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:03.163726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decbf2f-c2dd-4788-8c3f-bc478735b258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:03.164426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decbf2f-c2dd-4788-8c3f-bc478735b258 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:03.164713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decbf2f-c2dd-4788-8c3f-bc478735b258 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:03.164994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decbf2f-c2dd-4788-8c3f-bc478735b258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:03.165840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decbf2f-c2dd-4788-8c3f-bc478735b258 None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=812342d822f54aaf98e324d5d4e56753) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:03.166152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decbf2f-c2dd-4788-8c3f-bc478735b258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:03.167629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decbf2f-c2dd-4788-8c3f-bc478735b258 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:03.172203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 359/717] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:03.177967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee0029-a6b3-4fec-bafa-bb7662fa1141 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:03.182008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 359/718] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:02 2022] GET /identity/v3/role_assignments?user.id=812342d822f54aaf98e324d5d4e56753&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 608 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:03.208764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:03.357797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee0029-a6b3-4fec-bafa-bb7662fa1141 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:03.400628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fdd85-0b5c-4e1f-8427-be11cce6087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:03.401507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fdd85-0b5c-4e1f-8427-be11cce6087e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:03.401838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fdd85-0b5c-4e1f-8427-be11cce6087e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:03.402115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fdd85-0b5c-4e1f-8427-be11cce6087e None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:03.402882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fdd85-0b5c-4e1f-8427-be11cce6087e None admin] RBAC: Adding query filter params (type=network) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:03.403192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fdd85-0b5c-4e1f-8427-be11cce6087e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:03.404628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fdd85-0b5c-4e1f-8427-be11cce6087e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:03.412591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 360/719] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:04.169429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b5474-b996-4f6e-97aa-9da43f0e2fca None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:04.169984 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b5474-b996-4f6e-97aa-9da43f0e2fca None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:04.170302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b5474-b996-4f6e-97aa-9da43f0e2fca None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:04.171860 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 360/720] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:04.178233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3790a1e-b4af-4ab9-ac0d-8c8e971d89bf None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:04.178716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3790a1e-b4af-4ab9-ac0d-8c8e971d89bf None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:04.178791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3790a1e-b4af-4ab9-ac0d-8c8e971d89bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:04.203227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-b3790a1e-b4af-4ab9-ac0d-8c8e971d89bf None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:04.203227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b3790a1e-b4af-4ab9-ac0d-8c8e971d89bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:04.309407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 361/721] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:41:04.339719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b5474-b996-4f6e-97aa-9da43f0e2fca None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:04.486075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b5474-b996-4f6e-97aa-9da43f0e2fca None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:04.528081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27039eb-2164-434a-92a9-b02359e89fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:04.529164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27039eb-2164-434a-92a9-b02359e89fde None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:04.529455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27039eb-2164-434a-92a9-b02359e89fde None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:04.529732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27039eb-2164-434a-92a9-b02359e89fde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:04.546099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27039eb-2164-434a-92a9-b02359e89fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-b27039eb-2164-434a-92a9-b02359e89fde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:04.554485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:04.555473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:04.556358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:04.560408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 361/722] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:41:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:04.561980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3790a1e-b4af-4ab9-ac0d-8c8e971d89bf None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:04.572829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:04.692044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3790a1e-b4af-4ab9-ac0d-8c8e971d89bf None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:04.739974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae85d0-ae96-4ce4-bf45-05e66e8c4532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:04.740596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae85d0-ae96-4ce4-bf45-05e66e8c4532 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:04.740882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae85d0-ae96-4ce4-bf45-05e66e8c4532 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:04.741161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae85d0-ae96-4ce4-bf45-05e66e8c4532 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:04.741918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae85d0-ae96-4ce4-bf45-05e66e8c4532 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:04.742233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae85d0-ae96-4ce4-bf45-05e66e8c4532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:04.743651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae85d0-ae96-4ce4-bf45-05e66e8c4532 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:04.753504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 362/723] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:41:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:04.761495 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27039eb-2164-434a-92a9-b02359e89fde None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:04.792071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:04.886793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27039eb-2164-434a-92a9-b02359e89fde None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:04.924251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f6875-2923-4bf5-9537-f76f60271177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:04.925092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f6875-2923-4bf5-9537-f76f60271177 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:04.925381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f6875-2923-4bf5-9537-f76f60271177 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:04.925660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f6875-2923-4bf5-9537-f76f60271177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:04.938362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f6875-2923-4bf5-9537-f76f60271177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:04.939848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f6875-2923-4bf5-9537-f76f60271177 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-380f6875-2923-4bf5-9537-f76f60271177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:04.959288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:04.961487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:04.963417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:04.963417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:04.963417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:04.963417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:04.963417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 362/724] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:04.969503 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae85d0-ae96-4ce4-bf45-05e66e8c4532 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:05.061678 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:05.096230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae85d0-ae96-4ce4-bf45-05e66e8c4532 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:05.154250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761dbe5-b4d0-4f8b-bef6-731e2351e4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:05.154993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761dbe5-b4d0-4f8b-bef6-731e2351e4c6 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.155287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761dbe5-b4d0-4f8b-bef6-731e2351e4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.155573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761dbe5-b4d0-4f8b-bef6-731e2351e4c6 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.156335 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761dbe5-b4d0-4f8b-bef6-731e2351e4c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:05.156660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761dbe5-b4d0-4f8b-bef6-731e2351e4c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:05.158049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761dbe5-b4d0-4f8b-bef6-731e2351e4c6 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:05.174023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 363/725] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:05.179289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f6875-2923-4bf5-9537-f76f60271177 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:05.270604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:05.307568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f6875-2923-4bf5-9537-f76f60271177 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:05.344452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44157af-9370-4010-8b05-f1f4ef55b433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:05.345183 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44157af-9370-4010-8b05-f1f4ef55b433 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.345468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44157af-9370-4010-8b05-f1f4ef55b433 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.345742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44157af-9370-4010-8b05-f1f4ef55b433 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.354346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44157af-9370-4010-8b05-f1f4ef55b433 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:05.355884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44157af-9370-4010-8b05-f1f4ef55b433 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-f44157af-9370-4010-8b05-f1f4ef55b433 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:05.375576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:05.377612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:05.378797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:05.378797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:05.378797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:05.378797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:05.378797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:05.378797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:05.378797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:05.378797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:05.378797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:41:05.378797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:05.378797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 363/726] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jul 19 20:41:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:05.385503 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761dbe5-b4d0-4f8b-bef6-731e2351e4c6 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:05.413950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9faa0b-1456-4fc9-b7f4-8d4c98632e57 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.414278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9faa0b-1456-4fc9-b7f4-8d4c98632e57 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.414498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9faa0b-1456-4fc9-b7f4-8d4c98632e57 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.415905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 364/727] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:05.421370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d324f28-9767-4fc1-bb37-76c952fb1c7b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.421697 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d324f28-9767-4fc1-bb37-76c952fb1c7b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.421921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d324f28-9767-4fc1-bb37-76c952fb1c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.450560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-2d324f28-9767-4fc1-bb37-76c952fb1c7b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:05.451841 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2d324f28-9767-4fc1-bb37-76c952fb1c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:05.531016 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 365/728] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 20:41:05.568363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d324f28-9767-4fc1-bb37-76c952fb1c7b None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:05.569857 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761dbe5-b4d0-4f8b-bef6-731e2351e4c6 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:05.614054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:05.626433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17032a36-fa5b-4a54-9d57-19eca7cb2f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:05.627128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17032a36-fa5b-4a54-9d57-19eca7cb2f80 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.627410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17032a36-fa5b-4a54-9d57-19eca7cb2f80 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.627719 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17032a36-fa5b-4a54-9d57-19eca7cb2f80 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.628613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17032a36-fa5b-4a54-9d57-19eca7cb2f80 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:05.628927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17032a36-fa5b-4a54-9d57-19eca7cb2f80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:05.631041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17032a36-fa5b-4a54-9d57-19eca7cb2f80 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:05.654282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 364/729] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jul 19 20:41:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:05.656359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735196-08b2-4abd-8ab2-855bfd2d3a6f None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.656804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735196-08b2-4abd-8ab2-855bfd2d3a6f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.657031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:05.657238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66735196-08b2-4abd-8ab2-855bfd2d3a6f None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.659784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 365/730] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:05.661785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66735196-08b2-4abd-8ab2-855bfd2d3a6f None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:05.730979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d324f28-9767-4fc1-bb37-76c952fb1c7b None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:05.775654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b27032-682d-4c7c-b50b-4d774074826f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:05.776625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b27032-682d-4c7c-b50b-4d774074826f None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.776912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b27032-682d-4c7c-b50b-4d774074826f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.777189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b27032-682d-4c7c-b50b-4d774074826f None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.778125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b27032-682d-4c7c-b50b-4d774074826f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:05.779766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b27032-682d-4c7c-b50b-4d774074826f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:05.799049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 366/731] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:41:05 2022] POST /identity/v3/services => generated 251 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:05.800607 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad7fe-8d56-4d96-99dd-e201400b89ba None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.801538 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad7fe-8d56-4d96-99dd-e201400b89ba None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.801879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad7fe-8d56-4d96-99dd-e201400b89ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.821575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-b34ad7fe-8d56-4d96-99dd-e201400b89ba None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:05.822837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b34ad7fe-8d56-4d96-99dd-e201400b89ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:05.842634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66735196-08b2-4abd-8ab2-855bfd2d3a6f None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:05.934311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011d08f-bd20-4dc2-992b-b5b348824811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:05.935131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011d08f-bd20-4dc2-992b-b5b348824811 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.935528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011d08f-bd20-4dc2-992b-b5b348824811 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.935921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011d08f-bd20-4dc2-992b-b5b348824811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.947738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 367/732] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:41:05.952671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011d08f-bd20-4dc2-992b-b5b348824811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:05.954188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011d08f-bd20-4dc2-992b-b5b348824811 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:05.954391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d27c29-7353-4ccd-9f31-4afb6a6203f7 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.954984 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d27c29-7353-4ccd-9f31-4afb6a6203f7 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.955479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d27c29-7353-4ccd-9f31-4afb6a6203f7 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.958510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 368/733] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:05 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:05.964618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8327d-052d-4c49-ac71-b8d8d6b6c60d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:05.965139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8327d-052d-4c49-ac71-b8d8d6b6c60d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:05.965864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8327d-052d-4c49-ac71-b8d8d6b6c60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-f011d08f-bd20-4dc2-992b-b5b348824811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:05.983196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:05.984286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:05.985471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:05.985471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:05.985471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:05.985471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:05.985471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 366/734] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:05.993396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011d08f-bd20-4dc2-992b-b5b348824811 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:05.999807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-a9a8327d-052d-4c49-ac71-b8d8d6b6c60d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:06.000436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a9a8327d-052d-4c49-ac71-b8d8d6b6c60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:06.040881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:06.104591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:06.156433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 369/735] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 20:41:06.158658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8327d-052d-4c49-ac71-b8d8d6b6c60d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:06.159603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011d08f-bd20-4dc2-992b-b5b348824811 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:06.213480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9740a-1fc2-46bd-ae09-7cb4674ec892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:06.214267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9740a-1fc2-46bd-ae09-7cb4674ec892 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:06.214607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9740a-1fc2-46bd-ae09-7cb4674ec892 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:06.214939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9740a-1fc2-46bd-ae09-7cb4674ec892 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:06.231327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd9740a-1fc2-46bd-ae09-7cb4674ec892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-1bd9740a-1fc2-46bd-ae09-7cb4674ec892 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:06.240339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:06.241353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:41:06.242207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:06.245600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 367/736] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:41:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:06.246921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9740a-1fc2-46bd-ae09-7cb4674ec892 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:06.312520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8327d-052d-4c49-ac71-b8d8d6b6c60d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:06.366374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10aff38-406a-4aff-8120-0c24c529afe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:06.367570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aff38-406a-4aff-8120-0c24c529afe9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:06.368080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aff38-406a-4aff-8120-0c24c529afe9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:06.368978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aff38-406a-4aff-8120-0c24c529afe9 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:06.370692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10aff38-406a-4aff-8120-0c24c529afe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:06.371253 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10aff38-406a-4aff-8120-0c24c529afe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:06.373317 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10aff38-406a-4aff-8120-0c24c529afe9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:06.384002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:06.397009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9740a-1fc2-46bd-ae09-7cb4674ec892 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:06.398185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 370/737] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:06.399346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10aff38-406a-4aff-8120-0c24c529afe9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:06.451247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ee037-fc92-45a0-84a4-47dcc7a71f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:06.452839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ee037-fc92-45a0-84a4-47dcc7a71f1f None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:06.454314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ee037-fc92-45a0-84a4-47dcc7a71f1f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:06.454314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ee037-fc92-45a0-84a4-47dcc7a71f1f None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:06.455737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ee037-fc92-45a0-84a4-47dcc7a71f1f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:06.458445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ee037-fc92-45a0-84a4-47dcc7a71f1f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:06.462300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:06.482646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 368/738] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:41:06 2022] POST /identity/v3/services => generated 240 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:06.483582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ee037-fc92-45a0-84a4-47dcc7a71f1f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:06.539582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10aff38-406a-4aff-8120-0c24c529afe9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:06.587878 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88833c-ea6a-45f3-8bda-5491efb2a60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:06.589320 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88833c-ea6a-45f3-8bda-5491efb2a60f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:06.590001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88833c-ea6a-45f3-8bda-5491efb2a60f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:06.590652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88833c-ea6a-45f3-8bda-5491efb2a60f None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:06.592248 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88833c-ea6a-45f3-8bda-5491efb2a60f None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:06.593016 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88833c-ea6a-45f3-8bda-5491efb2a60f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:06.595739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88833c-ea6a-45f3-8bda-5491efb2a60f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:06.613471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 371/739] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:41:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:06.622713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88833c-ea6a-45f3-8bda-5491efb2a60f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:06.643185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ee037-fc92-45a0-84a4-47dcc7a71f1f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:06.708247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd901239-6aa6-4acc-b3b4-7fa52a3d4faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:06.709331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd901239-6aa6-4acc-b3b4-7fa52a3d4faf None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:06.709607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd901239-6aa6-4acc-b3b4-7fa52a3d4faf None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:06.709876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd901239-6aa6-4acc-b3b4-7fa52a3d4faf None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:06.719865 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd901239-6aa6-4acc-b3b4-7fa52a3d4faf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:06.722059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd901239-6aa6-4acc-b3b4-7fa52a3d4faf None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-cd901239-6aa6-4acc-b3b4-7fa52a3d4faf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:06.743783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:06.745823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:06.747843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:06.747843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:06.747843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:06.747843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:06.747843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:06.747843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:06.747843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:06.747843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:06.747843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 369/740] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:06.747843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38cb7c-46b7-4ba3-b391-c08fdccb5f1a None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:06.747843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38cb7c-46b7-4ba3-b391-c08fdccb5f1a None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:06.749479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38cb7c-46b7-4ba3-b391-c08fdccb5f1a None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:06.752864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 370/741] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:06.756045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f3524-3ebf-421d-8c16-fb95d5a3e378 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:06.756359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f3524-3ebf-421d-8c16-fb95d5a3e378 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:06.756756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f3524-3ebf-421d-8c16-fb95d5a3e378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:06.781372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-7a8f3524-3ebf-421d-8c16-fb95d5a3e378 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:06.781788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7a8f3524-3ebf-421d-8c16-fb95d5a3e378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:06.796612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:06.798935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:06.804021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88833c-ea6a-45f3-8bda-5491efb2a60f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:06.850579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d40215-cb48-42ed-acac-d0f7c69ceab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:06.851760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d40215-cb48-42ed-acac-d0f7c69ceab4 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:06.852239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d40215-cb48-42ed-acac-d0f7c69ceab4 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:06.852797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d40215-cb48-42ed-acac-d0f7c69ceab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:06.871240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d40215-cb48-42ed-acac-d0f7c69ceab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:06.873553 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d40215-cb48-42ed-acac-d0f7c69ceab4 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-49d40215-cb48-42ed-acac-d0f7c69ceab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:06.896472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:06.905153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:06.908039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:06.908039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:06.908039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:06.908039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:06.908039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 372/742] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:06.908039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d40215-cb48-42ed-acac-d0f7c69ceab4 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:06.920824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 371/743] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:06.926751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f3524-3ebf-421d-8c16-fb95d5a3e378 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:07.030571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d40215-cb48-42ed-acac-d0f7c69ceab4 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:07.077560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdca98-3aa0-4313-90b7-b95fced343a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:07.078251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdca98-3aa0-4313-90b7-b95fced343a4 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:07.078535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdca98-3aa0-4313-90b7-b95fced343a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:07.078812 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdca98-3aa0-4313-90b7-b95fced343a4 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:07.079635 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdca98-3aa0-4313-90b7-b95fced343a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:07.079947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdca98-3aa0-4313-90b7-b95fced343a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:07.081541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdca98-3aa0-4313-90b7-b95fced343a4 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:07.096459 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 373/744] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:07.097350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdca98-3aa0-4313-90b7-b95fced343a4 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:07.120722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:07.232185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f3524-3ebf-421d-8c16-fb95d5a3e378 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:07.233590 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdca98-3aa0-4313-90b7-b95fced343a4 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:07.277624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61ac6a-6d63-4bd5-a950-f6db8dbcefff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:07.278670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ac6a-6d63-4bd5-a950-f6db8dbcefff None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:07.279180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ac6a-6d63-4bd5-a950-f6db8dbcefff None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:07.279660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61ac6a-6d63-4bd5-a950-f6db8dbcefff None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:07.280705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61ac6a-6d63-4bd5-a950-f6db8dbcefff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:07.281368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61ac6a-6d63-4bd5-a950-f6db8dbcefff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:07.283096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61ac6a-6d63-4bd5-a950-f6db8dbcefff None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:07.293068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8833135-bd06-41f8-97de-e5b78fbf1449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:07.293905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8833135-bd06-41f8-97de-e5b78fbf1449 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:07.294190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8833135-bd06-41f8-97de-e5b78fbf1449 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:07.294469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8833135-bd06-41f8-97de-e5b78fbf1449 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:07.300972 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 372/745] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:07.302631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61ac6a-6d63-4bd5-a950-f6db8dbcefff None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:07.311412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8833135-bd06-41f8-97de-e5b78fbf1449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:07.314947 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-b8833135-bd06-41f8-97de-e5b78fbf1449 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:07.324019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:07.325558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:07.326439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:07.330975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 374/746] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:41:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:07.331951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8833135-bd06-41f8-97de-e5b78fbf1449 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:07.427273 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:07.436446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61ac6a-6d63-4bd5-a950-f6db8dbcefff None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:07.475764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8833135-bd06-41f8-97de-e5b78fbf1449 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:07.478438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ea254-7ac3-48f4-bbab-3648f2321f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:07.480525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ea254-7ac3-48f4-bbab-3648f2321f63 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:07.480924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ea254-7ac3-48f4-bbab-3648f2321f63 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:07.481325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ea254-7ac3-48f4-bbab-3648f2321f63 None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/928b8b2b510b4c939b37b2c0dfa57060/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:07.483773 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ea254-7ac3-48f4-bbab-3648f2321f63 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=928b8b2b510b4c939b37b2c0dfa57060, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:07.485911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ea254-7ac3-48f4-bbab-3648f2321f63 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:07.506314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 373/747] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:41:07 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/928b8b2b510b4c939b37b2c0dfa57060/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:41:07.508181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ea254-7ac3-48f4-bbab-3648f2321f63 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:07.522720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce8d15-8555-4846-b238-fad98318e16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:07.524180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce8d15-8555-4846-b238-fad98318e16b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:07.524913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce8d15-8555-4846-b238-fad98318e16b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:07.525422 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce8d15-8555-4846-b238-fad98318e16b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:07.546060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce8d15-8555-4846-b238-fad98318e16b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:07.548534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce8d15-8555-4846-b238-fad98318e16b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-c3ce8d15-8555-4846-b238-fad98318e16b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:07.578576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:07.579591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:07.580640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:07.580640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:07.580640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:07.580640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:07.580640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:07.580640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:07.580640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:07.580640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:07.580640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:07.580640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:07.590103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:07.591674 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 375/748] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jul 19 20:41:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:07.597019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce8d15-8555-4846-b238-fad98318e16b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:07.656682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ea254-7ac3-48f4-bbab-3648f2321f63 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:07.757301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8011737-a0fb-4ca6-8998-b75693170b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:07.759198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8011737-a0fb-4ca6-8998-b75693170b7a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:07.759688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8011737-a0fb-4ca6-8998-b75693170b7a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:07.761662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8011737-a0fb-4ca6-8998-b75693170b7a None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:07.762678 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8011737-a0fb-4ca6-8998-b75693170b7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:07.764257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8011737-a0fb-4ca6-8998-b75693170b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:07.765882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8011737-a0fb-4ca6-8998-b75693170b7a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:07.789136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 374/749] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:41:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:07.799934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8011737-a0fb-4ca6-8998-b75693170b7a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:07.808522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce8d15-8555-4846-b238-fad98318e16b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:07.839595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:07.845956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cd432-536b-496b-a78f-9c996622745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:07.846984 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd432-536b-496b-a78f-9c996622745e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:07.847463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd432-536b-496b-a78f-9c996622745e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:07.848554 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd432-536b-496b-a78f-9c996622745e None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:07.849602 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cd432-536b-496b-a78f-9c996622745e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:07.850112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cd432-536b-496b-a78f-9c996622745e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:07.851766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cd432-536b-496b-a78f-9c996622745e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:07.873062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 376/750] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jul 19 20:41:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:07.882411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cd432-536b-496b-a78f-9c996622745e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:07.963641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8011737-a0fb-4ca6-8998-b75693170b7a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:07.996097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:08.006464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca3306-5667-4b8a-9699-32c5e5d003f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:08.007676 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3306-5667-4b8a-9699-32c5e5d003f9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:08.008225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3306-5667-4b8a-9699-32c5e5d003f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:08.008941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3306-5667-4b8a-9699-32c5e5d003f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:08.024008 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca3306-5667-4b8a-9699-32c5e5d003f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:08.025847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca3306-5667-4b8a-9699-32c5e5d003f9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-d1ca3306-5667-4b8a-9699-32c5e5d003f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:08.051185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:08.052843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:08.054416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:08.054416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:08.054416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:08.054416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:08.057581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 375/751] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:08.065381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca3306-5667-4b8a-9699-32c5e5d003f9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:08.110609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cd432-536b-496b-a78f-9c996622745e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:08.188029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4d1e9-fb25-4f59-bce7-3bd5c0980468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:08.188831 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4d1e9-fb25-4f59-bce7-3bd5c0980468 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:08.189124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4d1e9-fb25-4f59-bce7-3bd5c0980468 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:08.189408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4d1e9-fb25-4f59-bce7-3bd5c0980468 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:08.237123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca3306-5667-4b8a-9699-32c5e5d003f9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:08.270324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:08.292519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4d1e9-fb25-4f59-bce7-3bd5c0980468 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:08.294138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4d1e9-fb25-4f59-bce7-3bd5c0980468 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:08.295220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8e9a5-fb81-44cd-aacb-c14ce8fa1d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:08.296192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8e9a5-fb81-44cd-aacb-c14ce8fa1d96 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:08.296784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8e9a5-fb81-44cd-aacb-c14ce8fa1d96 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:08.297264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8e9a5-fb81-44cd-aacb-c14ce8fa1d96 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:08.298192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8e9a5-fb81-44cd-aacb-c14ce8fa1d96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:08.298716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8e9a5-fb81-44cd-aacb-c14ce8fa1d96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:08.300287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8e9a5-fb81-44cd-aacb-c14ce8fa1d96 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:08.315198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 376/752] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:08.321694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8e9a5-fb81-44cd-aacb-c14ce8fa1d96 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:08.372848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:08.436820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 377/753] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:41:08.439646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ab54c-96d5-4dd1-8b94-d704fc380db2 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:08.440147 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ab54c-96d5-4dd1-8b94-d704fc380db2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:08.440680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ab54c-96d5-4dd1-8b94-d704fc380db2 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:08.442916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 378/754] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:08.444738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ab54c-96d5-4dd1-8b94-d704fc380db2 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:08.451189 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8e9a5-fb81-44cd-aacb-c14ce8fa1d96 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:08.489544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb87464-8081-439b-ab48-d983a847f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:08.490364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb87464-8081-439b-ab48-d983a847f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:08.490701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb87464-8081-439b-ab48-d983a847f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:08.490986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb87464-8081-439b-ab48-d983a847f7d4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:08.501572 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb87464-8081-439b-ab48-d983a847f7d4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:08.504981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb87464-8081-439b-ab48-d983a847f7d4 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-bbb87464-8081-439b-ab48-d983a847f7d4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:08.528184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:08.529231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:08.530218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:08.530218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:08.530218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:08.530218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:08.530218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:08.530218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:08.530218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:08.530218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:08.530218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:41:08.530218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:08.534232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 377/755] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:41:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:08.535681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e27aff-7ac7-4825-b550-b69d093b873a None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:08.535978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e27aff-7ac7-4825-b550-b69d093b873a None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:08.536265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e27aff-7ac7-4825-b550-b69d093b873a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:08.554725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-36e27aff-7ac7-4825-b550-b69d093b873a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:08.555976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-36e27aff-7ac7-4825-b550-b69d093b873a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:08.567966 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:08.569935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ab54c-96d5-4dd1-8b94-d704fc380db2 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:08.610460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bce06d-889b-4cf8-8a75-7fa40e1d55ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:08.611437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bce06d-889b-4cf8-8a75-7fa40e1d55ac None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:08.611723 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bce06d-889b-4cf8-8a75-7fa40e1d55ac None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:08.611992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bce06d-889b-4cf8-8a75-7fa40e1d55ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:08.626038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bce06d-889b-4cf8-8a75-7fa40e1d55ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:08.627528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bce06d-889b-4cf8-8a75-7fa40e1d55ac None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:08.644445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 378/756] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:08.645443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e27aff-7ac7-4825-b550-b69d093b873a None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-88bce06d-889b-4cf8-8a75-7fa40e1d55ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:08.649592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:08.651669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:08.653069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:08.653069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:08.653069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:08.653069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:08.653069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 379/757] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:08.665864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bce06d-889b-4cf8-8a75-7fa40e1d55ac None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:08.688898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:08.767553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e27aff-7ac7-4825-b550-b69d093b873a None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:08.805117 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f73077-3428-45f6-8753-b70147d71c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:08.806042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f73077-3428-45f6-8753-b70147d71c59 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:08.806405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f73077-3428-45f6-8753-b70147d71c59 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:08.807005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f73077-3428-45f6-8753-b70147d71c59 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:08.807945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f73077-3428-45f6-8753-b70147d71c59 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:08.808353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f73077-3428-45f6-8753-b70147d71c59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:08.809886 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f73077-3428-45f6-8753-b70147d71c59 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:08.821838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bce06d-889b-4cf8-8a75-7fa40e1d55ac None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:08.834701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 379/758] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jul 19 20:41:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:08.835725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f73077-3428-45f6-8753-b70147d71c59 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:08.861774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358bf28-dbd1-408b-baf6-8f0337cff453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:08.862433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358bf28-dbd1-408b-baf6-8f0337cff453 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:08.862714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358bf28-dbd1-408b-baf6-8f0337cff453 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:08.862988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358bf28-dbd1-408b-baf6-8f0337cff453 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:08.864335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358bf28-dbd1-408b-baf6-8f0337cff453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:08.865399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358bf28-dbd1-408b-baf6-8f0337cff453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:08.866772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358bf28-dbd1-408b-baf6-8f0337cff453 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:08.875497 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:08.884590 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 380/759] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:08.885730 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358bf28-dbd1-408b-baf6-8f0337cff453 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:08.933025 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:08.948073 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f73077-3428-45f6-8753-b70147d71c59 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:08.987058 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a60407-9014-4472-9d1f-d57ec334254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:08.988067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a60407-9014-4472-9d1f-d57ec334254c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:08.989040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a60407-9014-4472-9d1f-d57ec334254c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:08.989453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a60407-9014-4472-9d1f-d57ec334254c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:08.990222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a60407-9014-4472-9d1f-d57ec334254c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:08.991690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a60407-9014-4472-9d1f-d57ec334254c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-f5a60407-9014-4472-9d1f-d57ec334254c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:09.005154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:09.007198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:41:09.008867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:09.012627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 380/760] 10.222.0.85 () {56 vars in 1129 bytes} [Tue Jul 19 20:41:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:09.013622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a60407-9014-4472-9d1f-d57ec334254c None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:09.027760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358bf28-dbd1-408b-baf6-8f0337cff453 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:09.085524 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb530fc-64a4-4253-95fb-02e3f53befb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:09.086869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb530fc-64a4-4253-95fb-02e3f53befb0 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:09.087541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb530fc-64a4-4253-95fb-02e3f53befb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:09.088241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb530fc-64a4-4253-95fb-02e3f53befb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:09.172198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a60407-9014-4472-9d1f-d57ec334254c None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:09.185866 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:09.189777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb530fc-64a4-4253-95fb-02e3f53befb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:09.191567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb530fc-64a4-4253-95fb-02e3f53befb0 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:09.222102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39ec28-1337-419d-a240-875150dfbd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:09.223432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39ec28-1337-419d-a240-875150dfbd35 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:09.223722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39ec28-1337-419d-a240-875150dfbd35 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:09.224019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39ec28-1337-419d-a240-875150dfbd35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:09.231943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39ec28-1337-419d-a240-875150dfbd35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:09.233433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39ec28-1337-419d-a240-875150dfbd35 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-ec39ec28-1337-419d-a240-875150dfbd35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:09.249204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:09.250160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:09.251039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:09.251039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:09.251039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:09.251039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:09.251039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:09.251039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:09.251039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:09.251039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:09.255215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 381/761] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:09.256075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39ec28-1337-419d-a240-875150dfbd35 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:09.271980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:09.324388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 381/762] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:41:09.324388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee862d82-d6ad-40fd-b8a6-75021b08584d None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:09.324388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee862d82-d6ad-40fd-b8a6-75021b08584d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:09.324388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee862d82-d6ad-40fd-b8a6-75021b08584d None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:09.325004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 382/763] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:09.326592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee862d82-d6ad-40fd-b8a6-75021b08584d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:09.374934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39ec28-1337-419d-a240-875150dfbd35 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:09.416401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c0ce5-c1b8-49de-b6d5-90266329e7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:09.417576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c0ce5-c1b8-49de-b6d5-90266329e7d4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:09.417870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c0ce5-c1b8-49de-b6d5-90266329e7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:09.418149 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c0ce5-c1b8-49de-b6d5-90266329e7d4 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:09.419228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c0ce5-c1b8-49de-b6d5-90266329e7d4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:09.419546 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c0ce5-c1b8-49de-b6d5-90266329e7d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:09.421271 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c0ce5-c1b8-49de-b6d5-90266329e7d4 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:09.426468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 382/764] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jul 19 20:41:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:09.427195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c0ce5-c1b8-49de-b6d5-90266329e7d4 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:09.450712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:09.456982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee862d82-d6ad-40fd-b8a6-75021b08584d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:09.498392 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c66bb-fcb3-434e-bd3b-72ff9653d8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:09.499180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c66bb-fcb3-434e-bd3b-72ff9653d8e1 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:09.499464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c66bb-fcb3-434e-bd3b-72ff9653d8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:09.499747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c66bb-fcb3-434e-bd3b-72ff9653d8e1 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:09.500576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c66bb-fcb3-434e-bd3b-72ff9653d8e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:09.500891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c66bb-fcb3-434e-bd3b-72ff9653d8e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:09.502452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c66bb-fcb3-434e-bd3b-72ff9653d8e1 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:09.522104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 383/765] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:09.522104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51175e-8c52-4d37-8db2-06dff5d1f6f8 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:09.522104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51175e-8c52-4d37-8db2-06dff5d1f6f8 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:09.522104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51175e-8c52-4d37-8db2-06dff5d1f6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:09.532547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:09.539172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-fa51175e-8c52-4d37-8db2-06dff5d1f6f8 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:09.540807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fa51175e-8c52-4d37-8db2-06dff5d1f6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:09.565601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c0ce5-c1b8-49de-b6d5-90266329e7d4 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:09.607011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e8fb9-496b-4d0f-8554-176c3d45c6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:09.607822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e8fb9-496b-4d0f-8554-176c3d45c6ea None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:09.608133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e8fb9-496b-4d0f-8554-176c3d45c6ea None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:09.608569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e8fb9-496b-4d0f-8554-176c3d45c6ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:09.620214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e8fb9-496b-4d0f-8554-176c3d45c6ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:09.622072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e8fb9-496b-4d0f-8554-176c3d45c6ea None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:09.640436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 384/766] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:09.640436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51175e-8c52-4d37-8db2-06dff5d1f6f8 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-f29e8fb9-496b-4d0f-8554-176c3d45c6ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:09.646921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:09.647871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:09.648903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:09.648903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:09.648903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:09.648903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:09.648903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 383/767] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:09.649205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e8fb9-496b-4d0f-8554-176c3d45c6ea None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:09.768680 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e8fb9-496b-4d0f-8554-176c3d45c6ea None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:09.822429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:09.832789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1490a-ea9f-4c1e-bd48-b32b91aff5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:09.833472 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1490a-ea9f-4c1e-bd48-b32b91aff5f8 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:09.833761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1490a-ea9f-4c1e-bd48-b32b91aff5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:09.834529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1490a-ea9f-4c1e-bd48-b32b91aff5f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:09.841948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1490a-ea9f-4c1e-bd48-b32b91aff5f8 None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=e2f34a304e714f838d68dfdf214e4712) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:09.841948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1490a-ea9f-4c1e-bd48-b32b91aff5f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:09.841948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1490a-ea9f-4c1e-bd48-b32b91aff5f8 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:09.856956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 384/768] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:09 2022] GET /identity/v3/role_assignments?user.id=e2f34a304e714f838d68dfdf214e4712&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:09.856956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1490a-ea9f-4c1e-bd48-b32b91aff5f8 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:09.930322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:09.958185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51175e-8c52-4d37-8db2-06dff5d1f6f8 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:10.005344 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df82cff-60dc-4452-9cc3-5e061c34ccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:10.006230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df82cff-60dc-4452-9cc3-5e061c34ccbb None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.006614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df82cff-60dc-4452-9cc3-5e061c34ccbb None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.008178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df82cff-60dc-4452-9cc3-5e061c34ccbb None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.008178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df82cff-60dc-4452-9cc3-5e061c34ccbb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:10.008178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df82cff-60dc-4452-9cc3-5e061c34ccbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:10.010197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df82cff-60dc-4452-9cc3-5e061c34ccbb None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:10.016406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 385/769] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jul 19 20:41:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:10.017057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df82cff-60dc-4452-9cc3-5e061c34ccbb None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:10.018929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1490a-ea9f-4c1e-bd48-b32b91aff5f8 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:10.064941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9690-ce94-4485-bf75-7bac49aa8ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:10.065622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9690-ce94-4485-bf75-7bac49aa8ede None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.066072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9690-ce94-4485-bf75-7bac49aa8ede None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.066375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c9690-ce94-4485-bf75-7bac49aa8ede None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.068540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c9690-ce94-4485-bf75-7bac49aa8ede None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:10.068865 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c9690-ce94-4485-bf75-7bac49aa8ede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:10.070279 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c9690-ce94-4485-bf75-7bac49aa8ede None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:10.086490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 385/770] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:10.087448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9690-ce94-4485-bf75-7bac49aa8ede None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:10.142903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df82cff-60dc-4452-9cc3-5e061c34ccbb None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:10.153853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:10.185362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb00fdf-e42d-4166-945a-31470f238fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:10.186060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00fdf-e42d-4166-945a-31470f238fd0 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.186336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00fdf-e42d-4166-945a-31470f238fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.186612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00fdf-e42d-4166-945a-31470f238fd0 None admin] PATH_INFO: `/v3/services/network` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.187271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb00fdf-e42d-4166-945a-31470f238fd0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:10.188677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb00fdf-e42d-4166-945a-31470f238fd0 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-bfb00fdf-e42d-4166-945a-31470f238fd0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:10.204059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:10.208087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:10.209173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 386/771] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:10 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:10.211071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb00fdf-e42d-4166-945a-31470f238fd0 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:10.233079 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:10.245038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c9690-ce94-4485-bf75-7bac49aa8ede None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:10.285005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e86179-ef1d-4633-925d-bf7c4b697722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:10.286435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e86179-ef1d-4633-925d-bf7c4b697722 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.286876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e86179-ef1d-4633-925d-bf7c4b697722 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.287148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e86179-ef1d-4633-925d-bf7c4b697722 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.287958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e86179-ef1d-4633-925d-bf7c4b697722 None admin] RBAC: Adding query filter params (interface=public, service_id=3dd61c43976443218e9bd11bb20e95a3, region_id=RegionOne) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:10.288260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e86179-ef1d-4633-925d-bf7c4b697722 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:10.289759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e86179-ef1d-4633-925d-bf7c4b697722 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:10.305383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 386/772] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jul 19 20:41:10 2022] GET /identity/v3/endpoints?service_id=3dd61c43976443218e9bd11bb20e95a3&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:10.306979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e392-8e0c-4c9e-9ae8-715cdf3a8817 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.307276 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e392-8e0c-4c9e-9ae8-715cdf3a8817 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.307558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e392-8e0c-4c9e-9ae8-715cdf3a8817 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.312889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 387/773] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:10.314143 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1e392-8e0c-4c9e-9ae8-715cdf3a8817 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:10.341935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb00fdf-e42d-4166-945a-31470f238fd0 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:10.393004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eaf5dc-2cdf-452f-b73d-44fb758287a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:10.393807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaf5dc-2cdf-452f-b73d-44fb758287a0 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.394081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaf5dc-2cdf-452f-b73d-44fb758287a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.394354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eaf5dc-2cdf-452f-b73d-44fb758287a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.402140 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eaf5dc-2cdf-452f-b73d-44fb758287a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:10.403590 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eaf5dc-2cdf-452f-b73d-44fb758287a0 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-09eaf5dc-2cdf-452f-b73d-44fb758287a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:10.421475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:10.422454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:10.423359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:10.423359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:10.423359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:10.423359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:10.423359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:10.423359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:10.423359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:10.423359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:10.423359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 387/774] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:10.423857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eaf5dc-2cdf-452f-b73d-44fb758287a0 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:10.465491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1e392-8e0c-4c9e-9ae8-715cdf3a8817 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:10.483846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:10.506797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c597db0-53d3-49bc-aeab-d36bc2417dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:10.507537 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c597db0-53d3-49bc-aeab-d36bc2417dce None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.507821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c597db0-53d3-49bc-aeab-d36bc2417dce None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.508102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c597db0-53d3-49bc-aeab-d36bc2417dce None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.508851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c597db0-53d3-49bc-aeab-d36bc2417dce None admin] RBAC: Adding query filter params (name=network) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:10.509164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c597db0-53d3-49bc-aeab-d36bc2417dce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:10.510567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c597db0-53d3-49bc-aeab-d36bc2417dce None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:10.513857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:10.520121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 388/775] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:10.520121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8d210-b4e9-4602-9878-afd54fc86c97 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.520121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8d210-b4e9-4602-9878-afd54fc86c97 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.520121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8d210-b4e9-4602-9878-afd54fc86c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.538256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-8fa8d210-b4e9-4602-9878-afd54fc86c97 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:10.539856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8fa8d210-b4e9-4602-9878-afd54fc86c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:10.550378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eaf5dc-2cdf-452f-b73d-44fb758287a0 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:10.597006 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8605a95-ee8e-428a-86fa-4232a726aabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:10.598168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8605a95-ee8e-428a-86fa-4232a726aabc None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.598490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8605a95-ee8e-428a-86fa-4232a726aabc None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.598733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8605a95-ee8e-428a-86fa-4232a726aabc None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.599425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8605a95-ee8e-428a-86fa-4232a726aabc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:10.600868 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8605a95-ee8e-428a-86fa-4232a726aabc None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:10.608267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 388/776] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jul 19 20:41:10 2022] GET /identity/v3/services => generated 772 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:10.617396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8605a95-ee8e-428a-86fa-4232a726aabc None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:10.640320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 389/777] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:10.642216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8d210-b4e9-4602-9878-afd54fc86c97 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:10.775437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8605a95-ee8e-428a-86fa-4232a726aabc None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:10.786912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8d210-b4e9-4602-9878-afd54fc86c97 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:10.800185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:10.830861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc397e-b6fa-4cdf-bfd4-c16a698d46d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:10.831533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc397e-b6fa-4cdf-bfd4-c16a698d46d8 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.831817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc397e-b6fa-4cdf-bfd4-c16a698d46d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.832678 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc397e-b6fa-4cdf-bfd4-c16a698d46d8 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.833400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc397e-b6fa-4cdf-bfd4-c16a698d46d8 None admin] RBAC: Adding query filter params (type=network) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:10.833702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc397e-b6fa-4cdf-bfd4-c16a698d46d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:10.835103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc397e-b6fa-4cdf-bfd4-c16a698d46d8 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:10.841685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e4651-0446-45c3-a439-b7d87404dc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:10.842311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e4651-0446-45c3-a439-b7d87404dc25 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:10.843161 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e4651-0446-45c3-a439-b7d87404dc25 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:10.843161 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e4651-0446-45c3-a439-b7d87404dc25 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:10.843855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e4651-0446-45c3-a439-b7d87404dc25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:10.843959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e4651-0446-45c3-a439-b7d87404dc25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:10.845413 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e4651-0446-45c3-a439-b7d87404dc25 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:10.846323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 390/778] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:10.847447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc397e-b6fa-4cdf-bfd4-c16a698d46d8 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:10.862525 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 389/779] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:10.865110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e4651-0446-45c3-a439-b7d87404dc25 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:10.868142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:11.021357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e4651-0446-45c3-a439-b7d87404dc25 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:11.026248 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc397e-b6fa-4cdf-bfd4-c16a698d46d8 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:11.068296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253ffda-ba49-4d50-8cc6-7f6d74d1b9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:11.069332 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253ffda-ba49-4d50-8cc6-7f6d74d1b9f1 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:11.069709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253ffda-ba49-4d50-8cc6-7f6d74d1b9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:11.070443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253ffda-ba49-4d50-8cc6-7f6d74d1b9f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:11.071628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253ffda-ba49-4d50-8cc6-7f6d74d1b9f1 None admin] RBAC: Adding query filter params (interface=public, service_id=35cbe22202624f5882944a561b2114ed, region_id=RegionOne) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:11.072013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253ffda-ba49-4d50-8cc6-7f6d74d1b9f1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:11.074078 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253ffda-ba49-4d50-8cc6-7f6d74d1b9f1 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:11.079760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924ec68-b09d-45de-8619-4e64cfbc73f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:11.080787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924ec68-b09d-45de-8619-4e64cfbc73f4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:11.081224 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924ec68-b09d-45de-8619-4e64cfbc73f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:11.081593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924ec68-b09d-45de-8619-4e64cfbc73f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:11.083798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 390/780] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jul 19 20:41:10 2022] GET /identity/v3/endpoints?service_id=35cbe22202624f5882944a561b2114ed&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:11.084687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253ffda-ba49-4d50-8cc6-7f6d74d1b9f1 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:11.097607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924ec68-b09d-45de-8619-4e64cfbc73f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-c924ec68-b09d-45de-8619-4e64cfbc73f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:11.106438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:11.107590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:11.108487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:11.109140 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 391/781] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:11.112675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924ec68-b09d-45de-8619-4e64cfbc73f4 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:11.161918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:11.183969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:11.235853 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253ffda-ba49-4d50-8cc6-7f6d74d1b9f1 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:11.258192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924ec68-b09d-45de-8619-4e64cfbc73f4 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:11.279646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82b882-427d-48f4-8224-47b75d25afee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:11.280213 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82b882-427d-48f4-8224-47b75d25afee None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:11.280527 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82b882-427d-48f4-8224-47b75d25afee None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:11.280806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82b882-427d-48f4-8224-47b75d25afee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:11.282059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82b882-427d-48f4-8224-47b75d25afee None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=812342d822f54aaf98e324d5d4e56753) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:11.282376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82b882-427d-48f4-8224-47b75d25afee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:11.284128 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82b882-427d-48f4-8224-47b75d25afee None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:11.296578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 391/782] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:11 2022] GET /identity/v3/role_assignments?user.id=812342d822f54aaf98e324d5d4e56753&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:11.298143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82b882-427d-48f4-8224-47b75d25afee None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:11.328494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854ae38-51b8-4bd3-a714-873f5428a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:11.329457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854ae38-51b8-4bd3-a714-873f5428a474 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:11.329754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854ae38-51b8-4bd3-a714-873f5428a474 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:11.330035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854ae38-51b8-4bd3-a714-873f5428a474 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:11.330733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854ae38-51b8-4bd3-a714-873f5428a474 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:11.336357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7854ae38-51b8-4bd3-a714-873f5428a474 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:11.342120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 392/783] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jul 19 20:41:11 2022] GET /identity/v3/services => generated 772 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:11.435422 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82b882-427d-48f4-8224-47b75d25afee None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:11.472750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:11.480740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f5fa0-2c96-46b8-ac02-3642f5d42a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:11.481475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f5fa0-2c96-46b8-ac02-3642f5d42a94 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:11.481761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f5fa0-2c96-46b8-ac02-3642f5d42a94 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:11.482927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f5fa0-2c96-46b8-ac02-3642f5d42a94 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:11.482927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f5fa0-2c96-46b8-ac02-3642f5d42a94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:11.483095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f5fa0-2c96-46b8-ac02-3642f5d42a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:11.484553 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f5fa0-2c96-46b8-ac02-3642f5d42a94 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:11.495891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 392/784] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:41:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:11.501604 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854ae38-51b8-4bd3-a714-873f5428a474 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:11.626049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:11.631296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854ae38-51b8-4bd3-a714-873f5428a474 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:11.682704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ce89-f84f-42c6-af73-6b7ee972d348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:11.683528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ce89-f84f-42c6-af73-6b7ee972d348 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:11.683813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ce89-f84f-42c6-af73-6b7ee972d348 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:11.684091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ce89-f84f-42c6-af73-6b7ee972d348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:11.699754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495ce89-f84f-42c6-af73-6b7ee972d348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:11.701390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495ce89-f84f-42c6-af73-6b7ee972d348 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-d495ce89-f84f-42c6-af73-6b7ee972d348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:11.720870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:11.721870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:11.723335 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:11.723335 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:11.723335 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:11.723335 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:11.723335 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 393/785] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:11.728366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f5fa0-2c96-46b8-ac02-3642f5d42a94 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:11.848453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f5fa0-2c96-46b8-ac02-3642f5d42a94 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:11.885397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee0457-8c5d-4352-a5ea-320b433b2076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:11.885995 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee0457-8c5d-4352-a5ea-320b433b2076 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:11.886275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee0457-8c5d-4352-a5ea-320b433b2076 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:11.886548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee0457-8c5d-4352-a5ea-320b433b2076 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:11.887291 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee0457-8c5d-4352-a5ea-320b433b2076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:11.887817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee0457-8c5d-4352-a5ea-320b433b2076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:11.890218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee0457-8c5d-4352-a5ea-320b433b2076 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:11.908223 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 393/786] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:11.916604 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ce89-f84f-42c6-af73-6b7ee972d348 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:12.026696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ce89-f84f-42c6-af73-6b7ee972d348 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:12.064158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df439e-9686-42d2-86b3-987b1ef501a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:12.065017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df439e-9686-42d2-86b3-987b1ef501a0 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:12.065646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df439e-9686-42d2-86b3-987b1ef501a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:12.065926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df439e-9686-42d2-86b3-987b1ef501a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:12.074969 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df439e-9686-42d2-86b3-987b1ef501a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:12.076505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df439e-9686-42d2-86b3-987b1ef501a0 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-e1df439e-9686-42d2-86b3-987b1ef501a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:12.091808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:12.092883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:12.094533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:12.094533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:12.094533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:12.094533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:12.094533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:12.094533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:12.094533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:12.094533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:12.094533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:41:12.094533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:12.094533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 394/787] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jul 19 20:41:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:12.099780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee0457-8c5d-4352-a5ea-320b433b2076 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:12.158804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:12.209257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee0457-8c5d-4352-a5ea-320b433b2076 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:12.244257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb923c-f97f-489d-85b1-7bf29b3c5c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:12.244968 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb923c-f97f-489d-85b1-7bf29b3c5c7c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:12.245250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb923c-f97f-489d-85b1-7bf29b3c5c7c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:12.245526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb923c-f97f-489d-85b1-7bf29b3c5c7c None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:12.246365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb923c-f97f-489d-85b1-7bf29b3c5c7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:12.246673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb923c-f97f-489d-85b1-7bf29b3c5c7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:12.248107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb923c-f97f-489d-85b1-7bf29b3c5c7c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:12.267187 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 394/788] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jul 19 20:41:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:12.273283 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df439e-9686-42d2-86b3-987b1ef501a0 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:12.292759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:12.384020 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df439e-9686-42d2-86b3-987b1ef501a0 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:12.422853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60110832-9d3f-4ea8-bbed-a5addbb2747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:12.423688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60110832-9d3f-4ea8-bbed-a5addbb2747b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:12.424051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60110832-9d3f-4ea8-bbed-a5addbb2747b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:12.424456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60110832-9d3f-4ea8-bbed-a5addbb2747b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:12.432880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beae35a-3a7c-483a-980a-a815e138bd6f None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:12.433376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beae35a-3a7c-483a-980a-a815e138bd6f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:12.433839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beae35a-3a7c-483a-980a-a815e138bd6f None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:12.435612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 395/789] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:12.443324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b7f58-272a-4094-8550-a913df63074d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:12.443770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b7f58-272a-4094-8550-a913df63074d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:12.444118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b7f58-272a-4094-8550-a913df63074d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:12.473478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-843b7f58-272a-4094-8550-a913df63074d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:12.474711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-843b7f58-272a-4094-8550-a913df63074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:12.563286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 396/790] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:12.581457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60110832-9d3f-4ea8-bbed-a5addbb2747b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:12.583107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60110832-9d3f-4ea8-bbed-a5addbb2747b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:12.595942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:12.617184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b7f58-272a-4094-8550-a913df63074d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:12.696110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:12.732137 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 395/791] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:41:12.745999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60110832-9d3f-4ea8-bbed-a5addbb2747b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:12.834180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b7f58-272a-4094-8550-a913df63074d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:12.916216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8c033-32d2-4230-81ba-1b5f86cba52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:12.917055 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8c033-32d2-4230-81ba-1b5f86cba52b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:12.917334 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8c033-32d2-4230-81ba-1b5f86cba52b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:12.917607 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8c033-32d2-4230-81ba-1b5f86cba52b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:12.945316 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8c033-32d2-4230-81ba-1b5f86cba52b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:12.945695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60110832-9d3f-4ea8-bbed-a5addbb2747b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-14b8c033-32d2-4230-81ba-1b5f86cba52b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:12.957910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:12.959012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:12.960130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 397/792] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:41:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:12.962602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:12.967728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8c033-32d2-4230-81ba-1b5f86cba52b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:12.984130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4649a7-82ce-41f0-9cc2-f8fd11591088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:12.985058 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4649a7-82ce-41f0-9cc2-f8fd11591088 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:12.986289 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4649a7-82ce-41f0-9cc2-f8fd11591088 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:12.986598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4649a7-82ce-41f0-9cc2-f8fd11591088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:12.998100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4649a7-82ce-41f0-9cc2-f8fd11591088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:12.999569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4649a7-82ce-41f0-9cc2-f8fd11591088 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-5c4649a7-82ce-41f0-9cc2-f8fd11591088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:13.018967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:13.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:13.021681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:13.021681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:13.021681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:13.021681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:13.021681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 396/793] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:13.031199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4649a7-82ce-41f0-9cc2-f8fd11591088 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:13.151588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:13.162148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8c033-32d2-4230-81ba-1b5f86cba52b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:13.205440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845f81f-b6a0-4038-992e-7e51e4f729d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:13.206585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f81f-b6a0-4038-992e-7e51e4f729d2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:13.207664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f81f-b6a0-4038-992e-7e51e4f729d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:13.208184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f81f-b6a0-4038-992e-7e51e4f729d2 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:13.209077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845f81f-b6a0-4038-992e-7e51e4f729d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:13.209389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845f81f-b6a0-4038-992e-7e51e4f729d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:13.213074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845f81f-b6a0-4038-992e-7e51e4f729d2 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:13.215640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4649a7-82ce-41f0-9cc2-f8fd11591088 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:13.221373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 398/794] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:41:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:13.223751 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87151d94-d391-48bd-a2fa-3bf516ac1baa None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:13.224042 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87151d94-d391-48bd-a2fa-3bf516ac1baa None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:13.224316 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87151d94-d391-48bd-a2fa-3bf516ac1baa None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:13.230044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 399/795] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:13.231018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87151d94-d391-48bd-a2fa-3bf516ac1baa None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:13.260651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b9182-a080-435f-8f7e-984c9750b046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:13.261739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b9182-a080-435f-8f7e-984c9750b046 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:13.262255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b9182-a080-435f-8f7e-984c9750b046 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:13.262746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b9182-a080-435f-8f7e-984c9750b046 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:13.263692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b9182-a080-435f-8f7e-984c9750b046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:13.264219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b9182-a080-435f-8f7e-984c9750b046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:13.265877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b9182-a080-435f-8f7e-984c9750b046 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:13.279759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 397/796] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:13.281634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284c35-c068-4794-a9b4-f82a899443c4 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:13.282136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284c35-c068-4794-a9b4-f82a899443c4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:13.282615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284c35-c068-4794-a9b4-f82a899443c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:13.300796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-00284c35-c068-4794-a9b4-f82a899443c4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:13.302398 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-00284c35-c068-4794-a9b4-f82a899443c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:13.348378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:13.361601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87151d94-d391-48bd-a2fa-3bf516ac1baa None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:13.384693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 398/797] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:13.385749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284c35-c068-4794-a9b4-f82a899443c4 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:13.410696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c94a1b-960c-458e-bfd4-836126828294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:13.411789 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c94a1b-960c-458e-bfd4-836126828294 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:13.412264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c94a1b-960c-458e-bfd4-836126828294 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:13.412827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c94a1b-960c-458e-bfd4-836126828294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:13.427017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c94a1b-960c-458e-bfd4-836126828294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:13.429358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c94a1b-960c-458e-bfd4-836126828294 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-42c94a1b-960c-458e-bfd4-836126828294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:13.454711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:13.455922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:13.458093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:13.458093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:13.458093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:13.458093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:13.460810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 400/798] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:13.461683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c94a1b-960c-458e-bfd4-836126828294 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:13.480511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:13.553948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284c35-c068-4794-a9b4-f82a899443c4 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:13.596823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c94a1b-960c-458e-bfd4-836126828294 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:13.603702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3734474-5750-465f-b8f9-56d8ecb8ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:13.605343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3734474-5750-465f-b8f9-56d8ecb8ca9a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:13.605823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3734474-5750-465f-b8f9-56d8ecb8ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:13.606261 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3734474-5750-465f-b8f9-56d8ecb8ca9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:13.617612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3734474-5750-465f-b8f9-56d8ecb8ca9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:13.619812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3734474-5750-465f-b8f9-56d8ecb8ca9a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-b3734474-5750-465f-b8f9-56d8ecb8ca9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:13.640095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:13.641298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:13.642304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:13.642304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:13.642304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:13.642304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:13.642304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:13.642304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:13.642304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:13.642304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:13.646725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b19f6-31bf-42f0-b634-df6bc0245a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:13.647113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 399/799] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:13.648033 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b19f6-31bf-42f0-b634-df6bc0245a0c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:13.648157 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3734474-5750-465f-b8f9-56d8ecb8ca9a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:13.648686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b19f6-31bf-42f0-b634-df6bc0245a0c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:13.649114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b19f6-31bf-42f0-b634-df6bc0245a0c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:13.650051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b19f6-31bf-42f0-b634-df6bc0245a0c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:13.652133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b19f6-31bf-42f0-b634-df6bc0245a0c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:13.665943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-a49b19f6-31bf-42f0-b634-df6bc0245a0c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:13.671227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:13.672406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 20:41:13.673267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:13.680231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 401/800] 10.222.0.85 () {56 vars in 1129 bytes} [Tue Jul 19 20:41:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:13.681354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b19f6-31bf-42f0-b634-df6bc0245a0c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:13.786450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3734474-5750-465f-b8f9-56d8ecb8ca9a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:13.793127 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:13.822362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b19f6-31bf-42f0-b634-df6bc0245a0c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:13.829374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115602c-ecc0-41c8-98b4-af94fbecc5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:13.830117 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115602c-ecc0-41c8-98b4-af94fbecc5b3 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:13.830405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115602c-ecc0-41c8-98b4-af94fbecc5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:13.830681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115602c-ecc0-41c8-98b4-af94fbecc5b3 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:13.831745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115602c-ecc0-41c8-98b4-af94fbecc5b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:13.832653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115602c-ecc0-41c8-98b4-af94fbecc5b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:13.834492 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115602c-ecc0-41c8-98b4-af94fbecc5b3 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:13.855379 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 400/801] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:13.856646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115602c-ecc0-41c8-98b4-af94fbecc5b3 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:13.866763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e107cb-17be-4ca5-b069-7193c8c8f57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:13.868142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e107cb-17be-4ca5-b069-7193c8c8f57b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:13.868660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e107cb-17be-4ca5-b069-7193c8c8f57b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:13.869166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e107cb-17be-4ca5-b069-7193c8c8f57b None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:13.870189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e107cb-17be-4ca5-b069-7193c8c8f57b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:13.870732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e107cb-17be-4ca5-b069-7193c8c8f57b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:13.872438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e107cb-17be-4ca5-b069-7193c8c8f57b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:13.886318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 402/802] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:13.887574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e107cb-17be-4ca5-b069-7193c8c8f57b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:13.951177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:13.989495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115602c-ecc0-41c8-98b4-af94fbecc5b3 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:14.003123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e107cb-17be-4ca5-b069-7193c8c8f57b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:14.038513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37662b47-e95d-4ea4-af6f-6ec916de931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:14.039313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662b47-e95d-4ea4-af6f-6ec916de931a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.039693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662b47-e95d-4ea4-af6f-6ec916de931a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.040108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662b47-e95d-4ea4-af6f-6ec916de931a None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.041150 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37662b47-e95d-4ea4-af6f-6ec916de931a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:14.041467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37662b47-e95d-4ea4-af6f-6ec916de931a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:14.043090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37662b47-e95d-4ea4-af6f-6ec916de931a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:14.045036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcc789-acdd-403e-a8ef-222d12421302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:14.046131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcc789-acdd-403e-a8ef-222d12421302 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.046619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcc789-acdd-403e-a8ef-222d12421302 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.047096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcc789-acdd-403e-a8ef-222d12421302 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.054195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 401/803] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jul 19 20:41:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:14.055124 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37662b47-e95d-4ea4-af6f-6ec916de931a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:14.065900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcc789-acdd-403e-a8ef-222d12421302 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:14.068501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcc789-acdd-403e-a8ef-222d12421302 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:14.070907 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-1ebcc789-acdd-403e-a8ef-222d12421302 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:14.088693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:14.090467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 403/804] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jul 19 20:41:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:14.092243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcc789-acdd-403e-a8ef-222d12421302 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:14.222057 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37662b47-e95d-4ea4-af6f-6ec916de931a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:14.256321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcc789-acdd-403e-a8ef-222d12421302 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:14.258584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9506e2-4990-4fdb-808f-b9c238fea8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:14.259244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9506e2-4990-4fdb-808f-b9c238fea8df None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.259526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9506e2-4990-4fdb-808f-b9c238fea8df None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.259803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9506e2-4990-4fdb-808f-b9c238fea8df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.260688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9506e2-4990-4fdb-808f-b9c238fea8df None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=928b8b2b510b4c939b37b2c0dfa57060) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:14.261000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9506e2-4990-4fdb-808f-b9c238fea8df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:14.262434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9506e2-4990-4fdb-808f-b9c238fea8df None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:14.265618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:14.271012 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 402/805] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:14 2022] GET /identity/v3/role_assignments?user.id=928b8b2b510b4c939b37b2c0dfa57060&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:14.272475 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb372aa0-ef1e-449b-8941-461344eb2248 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.273029 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb372aa0-ef1e-449b-8941-461344eb2248 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.273601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb372aa0-ef1e-449b-8941-461344eb2248 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.275092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 403/806] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:14.279481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb372aa0-ef1e-449b-8941-461344eb2248 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:14.308852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b56df-2b3a-459a-8a96-b0f9f2f6f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:14.309781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b56df-2b3a-459a-8a96-b0f9f2f6f268 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.310222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b56df-2b3a-459a-8a96-b0f9f2f6f268 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.310641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b56df-2b3a-459a-8a96-b0f9f2f6f268 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.311713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b56df-2b3a-459a-8a96-b0f9f2f6f268 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:14.312291 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b56df-2b3a-459a-8a96-b0f9f2f6f268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:14.313843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b56df-2b3a-459a-8a96-b0f9f2f6f268 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:14.321567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 404/807] 10.222.0.85 () {56 vars in 1124 bytes} [Tue Jul 19 20:41:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:14.323311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7d69c-7038-4c5d-8bbc-8344e3761d88 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.323673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7d69c-7038-4c5d-8bbc-8344e3761d88 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.324024 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7d69c-7038-4c5d-8bbc-8344e3761d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.345082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-efc7d69c-7038-4c5d-8bbc-8344e3761d88 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:14.346547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-efc7d69c-7038-4c5d-8bbc-8344e3761d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:14.426539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb372aa0-ef1e-449b-8941-461344eb2248 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:14.442624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:14.461320 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 405/808] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:14.461511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7d69c-7038-4c5d-8bbc-8344e3761d88 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:14.470748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6e6f3-9215-4196-9fb8-5ec062ae0a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:14.472158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6e6f3-9215-4196-9fb8-5ec062ae0a07 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.473215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6e6f3-9215-4196-9fb8-5ec062ae0a07 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.474146 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6e6f3-9215-4196-9fb8-5ec062ae0a07 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.475961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6e6f3-9215-4196-9fb8-5ec062ae0a07 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:14.476844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6e6f3-9215-4196-9fb8-5ec062ae0a07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:14.480973 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6e6f3-9215-4196-9fb8-5ec062ae0a07 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:14.504310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 404/809] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jul 19 20:41:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:14.506008 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f12b6-6df6-4439-bf4a-a010b3f7cc6b None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.506302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f12b6-6df6-4439-bf4a-a010b3f7cc6b None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.506569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f12b6-6df6-4439-bf4a-a010b3f7cc6b None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.512409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 405/810] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:14.512409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f12b6-6df6-4439-bf4a-a010b3f7cc6b None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:14.583758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:14.593791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7d69c-7038-4c5d-8bbc-8344e3761d88 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:14.635685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f12b6-6df6-4439-bf4a-a010b3f7cc6b None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:14.636143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb65f6e-c4c3-4261-ac04-9498ae361cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:14.637020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb65f6e-c4c3-4261-ac04-9498ae361cef None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.637430 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb65f6e-c4c3-4261-ac04-9498ae361cef None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.637712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb65f6e-c4c3-4261-ac04-9498ae361cef None admin] PATH_INFO: `/v3/endpoints` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.638469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb65f6e-c4c3-4261-ac04-9498ae361cef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:14.639860 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb65f6e-c4c3-4261-ac04-9498ae361cef None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:14.674312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 406/811] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jul 19 20:41:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:14.674312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb65f6e-c4c3-4261-ac04-9498ae361cef None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:14.685406 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7120fa-d5db-4478-8c8a-fbc6868e8ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:14.686471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7120fa-d5db-4478-8c8a-fbc6868e8ad9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.686610 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7120fa-d5db-4478-8c8a-fbc6868e8ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.686874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7120fa-d5db-4478-8c8a-fbc6868e8ad9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.702409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7120fa-d5db-4478-8c8a-fbc6868e8ad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-8d7120fa-d5db-4478-8c8a-fbc6868e8ad9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:14.711296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:14.712609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:14.713762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 406/812] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:41:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:14.714477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c03e86-31d4-4506-9aef-feae4ce7d71b None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.714477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c03e86-31d4-4506-9aef-feae4ce7d71b None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.714599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c03e86-31d4-4506-9aef-feae4ce7d71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.737533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-80c03e86-31d4-4506-9aef-feae4ce7d71b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:14.738819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-80c03e86-31d4-4506-9aef-feae4ce7d71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:14.759291 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:14.826654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 407/813] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:14.827654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c03e86-31d4-4506-9aef-feae4ce7d71b None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:14.859186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb65f6e-c4c3-4261-ac04-9498ae361cef None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:14.910433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd4486-e336-4010-9afb-abf676454946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:14.911290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd4486-e336-4010-9afb-abf676454946 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:14.911578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd4486-e336-4010-9afb-abf676454946 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:14.911856 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd4486-e336-4010-9afb-abf676454946 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:14.916230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:14.928752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd4486-e336-4010-9afb-abf676454946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:14.930751 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd4486-e336-4010-9afb-abf676454946 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-7ccd4486-e336-4010-9afb-abf676454946 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:14.967917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:14.969516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:14.971084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:14.971084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:14.971084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:14.971084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:14.971084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 407/814] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:14.971084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd4486-e336-4010-9afb-abf676454946 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:15.021292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c03e86-31d4-4506-9aef-feae4ce7d71b None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:15.077657 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8f05a-8602-4023-986f-4e82992ef1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:15.078334 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8f05a-8602-4023-986f-4e82992ef1e5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:15.078623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8f05a-8602-4023-986f-4e82992ef1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:15.078900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8f05a-8602-4023-986f-4e82992ef1e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:15.079685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8f05a-8602-4023-986f-4e82992ef1e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:15.079998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8f05a-8602-4023-986f-4e82992ef1e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:15.081415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8f05a-8602-4023-986f-4e82992ef1e5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:15.096580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 408/815] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:15.097637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8f05a-8602-4023-986f-4e82992ef1e5 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:15.135610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:15.141078 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd4486-e336-4010-9afb-abf676454946 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:15.189092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523dced-58ee-439f-a6cc-149225aa03b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:15.190253 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523dced-58ee-439f-a6cc-149225aa03b2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:15.190772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523dced-58ee-439f-a6cc-149225aa03b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:15.191894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523dced-58ee-439f-a6cc-149225aa03b2 None admin] PATH_INFO: `/v3/services/network` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:15.192864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523dced-58ee-439f-a6cc-149225aa03b2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:15.195840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523dced-58ee-439f-a6cc-149225aa03b2 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-8523dced-58ee-439f-a6cc-149225aa03b2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:15.215067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:15.216694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 20:41:15.217583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:15.221586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 408/816] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:14 2022] GET /identity/v3/services/network => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:15.223022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523dced-58ee-439f-a6cc-149225aa03b2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:15.234132 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8f05a-8602-4023-986f-4e82992ef1e5 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:15.244100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:15.284869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18542eaf-13e6-4028-be5b-8264be1be26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:15.284869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18542eaf-13e6-4028-be5b-8264be1be26f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:15.284869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18542eaf-13e6-4028-be5b-8264be1be26f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:15.285629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18542eaf-13e6-4028-be5b-8264be1be26f None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:15.286504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18542eaf-13e6-4028-be5b-8264be1be26f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:15.286948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18542eaf-13e6-4028-be5b-8264be1be26f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:15.288634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18542eaf-13e6-4028-be5b-8264be1be26f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:15.304721 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 409/817] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:15.306044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18542eaf-13e6-4028-be5b-8264be1be26f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:15.366470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523dced-58ee-439f-a6cc-149225aa03b2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:15.403425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822cb57-6057-4a6a-ad16-b7ebc6f66def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:15.404500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822cb57-6057-4a6a-ad16-b7ebc6f66def None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:15.404928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822cb57-6057-4a6a-ad16-b7ebc6f66def None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:15.405527 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822cb57-6057-4a6a-ad16-b7ebc6f66def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:15.418525 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822cb57-6057-4a6a-ad16-b7ebc6f66def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:15.420546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822cb57-6057-4a6a-ad16-b7ebc6f66def None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:15.427285 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18542eaf-13e6-4028-be5b-8264be1be26f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-f822cb57-6057-4a6a-ad16-b7ebc6f66def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:15.440840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:15.442566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:15.443535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:15.443535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:15.443535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:15.443535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:15.450378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 409/818] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:15.451517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822cb57-6057-4a6a-ad16-b7ebc6f66def None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:15.467808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcab48-3a15-4e07-b2a5-2a971fbfe116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:15.468628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcab48-3a15-4e07-b2a5-2a971fbfe116 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:15.468983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcab48-3a15-4e07-b2a5-2a971fbfe116 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:15.469355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcab48-3a15-4e07-b2a5-2a971fbfe116 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:15.470232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcab48-3a15-4e07-b2a5-2a971fbfe116 None admin] RBAC: Adding query filter params (name=network) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:15.471003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcab48-3a15-4e07-b2a5-2a971fbfe116 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:15.472211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcab48-3a15-4e07-b2a5-2a971fbfe116 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:15.474120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:15.479471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 410/819] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:15.481500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcab48-3a15-4e07-b2a5-2a971fbfe116 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:15.517518 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:15.575014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822cb57-6057-4a6a-ad16-b7ebc6f66def None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:15.605944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcab48-3a15-4e07-b2a5-2a971fbfe116 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:15.615296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f14a2-656d-498d-88e7-229c3a8031d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:15.616460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f14a2-656d-498d-88e7-229c3a8031d2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:15.616956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f14a2-656d-498d-88e7-229c3a8031d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:15.617390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f14a2-656d-498d-88e7-229c3a8031d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:15.625981 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f14a2-656d-498d-88e7-229c3a8031d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:15.627754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f14a2-656d-498d-88e7-229c3a8031d2 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-613f14a2-656d-498d-88e7-229c3a8031d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:15.644027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:15.645153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:15.646047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:15.646047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:15.646047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:15.646047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:15.646047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:15.646047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:15.646047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:15.646047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:15.647356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3cc92-8eb8-45b1-8e4a-6d4a9f1ff37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:15.648085 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3cc92-8eb8-45b1-8e4a-6d4a9f1ff37d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:15.648477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3cc92-8eb8-45b1-8e4a-6d4a9f1ff37d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:15.648822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3cc92-8eb8-45b1-8e4a-6d4a9f1ff37d None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:15.649520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 410/820] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:15.649624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3cc92-8eb8-45b1-8e4a-6d4a9f1ff37d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:15.650682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f14a2-656d-498d-88e7-229c3a8031d2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:15.654991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3cc92-8eb8-45b1-8e4a-6d4a9f1ff37d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:15.656589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3cc92-8eb8-45b1-8e4a-6d4a9f1ff37d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:15.675308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 411/821] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:15.675308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3cc92-8eb8-45b1-8e4a-6d4a9f1ff37d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:15.748551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:15.774412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f14a2-656d-498d-88e7-229c3a8031d2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:15.782006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:15.795648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3cc92-8eb8-45b1-8e4a-6d4a9f1ff37d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:15.815834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6408cc9-b8b1-4307-ab4d-593d6333aa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:15.816893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408cc9-b8b1-4307-ab4d-593d6333aa94 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:15.817358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408cc9-b8b1-4307-ab4d-593d6333aa94 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:15.817952 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6408cc9-b8b1-4307-ab4d-593d6333aa94 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:15.818895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6408cc9-b8b1-4307-ab4d-593d6333aa94 None admin] RBAC: Adding query filter params (type=network) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:15.819462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6408cc9-b8b1-4307-ab4d-593d6333aa94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:15.821229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6408cc9-b8b1-4307-ab4d-593d6333aa94 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:15.827557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 411/822] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:15.828853 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6408cc9-b8b1-4307-ab4d-593d6333aa94 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:15.845343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925094ac-f8b2-44ec-ba24-e09124bb3b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:15.846089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925094ac-f8b2-44ec-ba24-e09124bb3b82 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:15.846329 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925094ac-f8b2-44ec-ba24-e09124bb3b82 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:15.846607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925094ac-f8b2-44ec-ba24-e09124bb3b82 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:15.847423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925094ac-f8b2-44ec-ba24-e09124bb3b82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:15.848591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925094ac-f8b2-44ec-ba24-e09124bb3b82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:15.849839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925094ac-f8b2-44ec-ba24-e09124bb3b82 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:15.869211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 412/823] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:15.870197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925094ac-f8b2-44ec-ba24-e09124bb3b82 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:15.947283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6408cc9-b8b1-4307-ab4d-593d6333aa94 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:15.977514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925094ac-f8b2-44ec-ba24-e09124bb3b82 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:15.995195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd64a69-c7e5-459c-802e-e34196894893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:15.997315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd64a69-c7e5-459c-802e-e34196894893 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:15.998133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd64a69-c7e5-459c-802e-e34196894893 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:15.998717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd64a69-c7e5-459c-802e-e34196894893 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.009066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd64a69-c7e5-459c-802e-e34196894893 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:16.010983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd64a69-c7e5-459c-802e-e34196894893 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:16.016721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:16.021984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4ab38-dce6-4673-bfda-3992bc7c0098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:16.023030 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4ab38-dce6-4673-bfda-3992bc7c0098 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:16.023501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4ab38-dce6-4673-bfda-3992bc7c0098 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:16.023971 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4ab38-dce6-4673-bfda-3992bc7c0098 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.025121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4ab38-dce6-4673-bfda-3992bc7c0098 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:16.026932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4ab38-dce6-4673-bfda-3992bc7c0098 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-cdd64a69-c7e5-459c-802e-e34196894893 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:16.033772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:16.035863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:16.037046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:16.037046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:16.037046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:16.037046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:16.037046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:16.037046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:16.037046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:16.037046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:16.037046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:41:16.037046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:16.043002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 412/824] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:41:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:16.044032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd64a69-c7e5-459c-802e-e34196894893 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:16.055671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 413/825] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jul 19 20:41:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:16.057643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4ab38-dce6-4673-bfda-3992bc7c0098 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:16.074980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:16.206412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4ab38-dce6-4673-bfda-3992bc7c0098 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:16.251759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6f046-ef34-42ff-9f4e-8f3e29a9cfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:16.252546 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6f046-ef34-42ff-9f4e-8f3e29a9cfe5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:16.252838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6f046-ef34-42ff-9f4e-8f3e29a9cfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:16.253111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6f046-ef34-42ff-9f4e-8f3e29a9cfe5 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.254565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6f046-ef34-42ff-9f4e-8f3e29a9cfe5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:16.254875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6f046-ef34-42ff-9f4e-8f3e29a9cfe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:16.256542 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6f046-ef34-42ff-9f4e-8f3e29a9cfe5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:16.277016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 414/826] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jul 19 20:41:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:16.283247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6f046-ef34-42ff-9f4e-8f3e29a9cfe5 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:16.296158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd64a69-c7e5-459c-802e-e34196894893 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:16.339044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd320a-e46a-40e6-97b5-8a3aa645dddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:16.339970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd320a-e46a-40e6-97b5-8a3aa645dddf None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:16.340260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd320a-e46a-40e6-97b5-8a3aa645dddf None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:16.340575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd320a-e46a-40e6-97b5-8a3aa645dddf None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/e2f34a304e714f838d68dfdf214e4712/roles/b2e0a9d2d13844e980a69f441f36d492` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.361488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd320a-e46a-40e6-97b5-8a3aa645dddf None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=e2f34a304e714f838d68dfdf214e4712, role_id=b2e0a9d2d13844e980a69f441f36d492)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:16.363664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd320a-e46a-40e6-97b5-8a3aa645dddf None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:16.374938 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:16.383046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 413/827] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:41:16 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/e2f34a304e714f838d68dfdf214e4712/roles/b2e0a9d2d13844e980a69f441f36d492 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:41:16.436549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6f046-ef34-42ff-9f4e-8f3e29a9cfe5 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:16.497023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937f4b3-904c-4437-9557-12ac7267a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:16.498183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937f4b3-904c-4437-9557-12ac7267a064 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:16.498639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937f4b3-904c-4437-9557-12ac7267a064 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:16.499063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937f4b3-904c-4437-9557-12ac7267a064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.518466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937f4b3-904c-4437-9557-12ac7267a064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:16.521167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937f4b3-904c-4437-9557-12ac7267a064 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-3937f4b3-904c-4437-9557-12ac7267a064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:16.551573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:16.552750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:16.553847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:16.553847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:16.553847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:16.553847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:16.560514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 415/828] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:16.562502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd320a-e46a-40e6-97b5-8a3aa645dddf None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:16.639331 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:16.647406 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4d3e4-8dcf-439c-92be-8930a8c1f463 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:16.647880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4d3e4-8dcf-439c-92be-8930a8c1f463 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:16.648382 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4d3e4-8dcf-439c-92be-8930a8c1f463 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.650925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 416/829] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:16.658117 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077a6be-993a-4ade-a75e-e632877c7ea0 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:16.658431 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077a6be-993a-4ade-a75e-e632877c7ea0 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:16.658859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077a6be-993a-4ade-a75e-e632877c7ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.684468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c077a6be-993a-4ade-a75e-e632877c7ea0 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:16.685958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c077a6be-993a-4ade-a75e-e632877c7ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:16.753329 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd320a-e46a-40e6-97b5-8a3aa645dddf None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:16.777819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 417/830] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:16.796252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46b4ac-b455-4305-9c84-706f91c76460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:16.797266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46b4ac-b455-4305-9c84-706f91c76460 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:16.797548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46b4ac-b455-4305-9c84-706f91c76460 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:16.797867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46b4ac-b455-4305-9c84-706f91c76460 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:16.798746 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46b4ac-b455-4305-9c84-706f91c76460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:16.799076 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46b4ac-b455-4305-9c84-706f91c76460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:16.801155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46b4ac-b455-4305-9c84-706f91c76460 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:16.803073 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077a6be-993a-4ade-a75e-e632877c7ea0 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:16.815048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 414/831] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:16.822409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46b4ac-b455-4305-9c84-706f91c76460 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:16.872626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:16.949798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077a6be-993a-4ade-a75e-e632877c7ea0 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:17.002644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36208e-17ab-42c6-bcd0-637600a0531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:17.003692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36208e-17ab-42c6-bcd0-637600a0531b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:17.004811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36208e-17ab-42c6-bcd0-637600a0531b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:17.005188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36208e-17ab-42c6-bcd0-637600a0531b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:17.005954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36208e-17ab-42c6-bcd0-637600a0531b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:17.007432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36208e-17ab-42c6-bcd0-637600a0531b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:17.008714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46b4ac-b455-4305-9c84-706f91c76460 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-da36208e-17ab-42c6-bcd0-637600a0531b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:17.025278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:17.027466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:17.033529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 418/832] 10.222.0.85 () {56 vars in 1132 bytes} [Tue Jul 19 20:41:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:17.037019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36208e-17ab-42c6-bcd0-637600a0531b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:17.050190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:17.087319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b123027-b3cd-4377-b537-b101d5ede4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:17.089458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b123027-b3cd-4377-b537-b101d5ede4a7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:17.090276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b123027-b3cd-4377-b537-b101d5ede4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:17.091070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b123027-b3cd-4377-b537-b101d5ede4a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:17.101855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b123027-b3cd-4377-b537-b101d5ede4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:17.103666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b123027-b3cd-4377-b537-b101d5ede4a7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-1b123027-b3cd-4377-b537-b101d5ede4a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:17.129964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:17.131648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:17.133302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:17.133302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:17.133302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:17.133302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:17.133302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:17.133302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:17.133302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:17.133302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:17.136475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 415/833] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:17.139066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e695b-3303-41f1-b237-5ab596f197a1 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:17.139757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e695b-3303-41f1-b237-5ab596f197a1 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:17.140806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e695b-3303-41f1-b237-5ab596f197a1 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:17.143552 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 416/834] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:17.145889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e695b-3303-41f1-b237-5ab596f197a1 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:17.180020 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36208e-17ab-42c6-bcd0-637600a0531b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:17.188897 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:17.223359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a812ef-5dc0-4891-bc13-5e86cc2bd903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:17.224123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a812ef-5dc0-4891-bc13-5e86cc2bd903 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:17.224514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a812ef-5dc0-4891-bc13-5e86cc2bd903 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:17.224866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a812ef-5dc0-4891-bc13-5e86cc2bd903 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:17.225588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a812ef-5dc0-4891-bc13-5e86cc2bd903 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:17.225989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a812ef-5dc0-4891-bc13-5e86cc2bd903 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:17.227375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a812ef-5dc0-4891-bc13-5e86cc2bd903 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:17.237746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 419/835] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jul 19 20:41:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:17.237746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e54b6-2cd1-4db9-80b0-b5f255e514d2 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:17.237746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e54b6-2cd1-4db9-80b0-b5f255e514d2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:17.237746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e54b6-2cd1-4db9-80b0-b5f255e514d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:17.254692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-9e3e54b6-2cd1-4db9-80b0-b5f255e514d2 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:17.255887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9e3e54b6-2cd1-4db9-80b0-b5f255e514d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:17.307485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e695b-3303-41f1-b237-5ab596f197a1 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:17.334757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 420/836] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:17.335978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e54b6-2cd1-4db9-80b0-b5f255e514d2 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:17.360485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cda700-e16a-4582-adcd-46761bc26022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:17.361584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda700-e16a-4582-adcd-46761bc26022 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:17.361872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda700-e16a-4582-adcd-46761bc26022 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:17.362150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda700-e16a-4582-adcd-46761bc26022 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:17.362956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cda700-e16a-4582-adcd-46761bc26022 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:17.363894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cda700-e16a-4582-adcd-46761bc26022 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:17.365519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cda700-e16a-4582-adcd-46761bc26022 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:17.381081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 417/837] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:17.382569 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cda700-e16a-4582-adcd-46761bc26022 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:17.438577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:17.485957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:17.490631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e54b6-2cd1-4db9-80b0-b5f255e514d2 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:17.540080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cda700-e16a-4582-adcd-46761bc26022 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:17.540703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f35480-2db2-4f0c-95ad-6c34f7ae4b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:17.541611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f35480-2db2-4f0c-95ad-6c34f7ae4b08 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:17.542041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f35480-2db2-4f0c-95ad-6c34f7ae4b08 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:17.542445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f35480-2db2-4f0c-95ad-6c34f7ae4b08 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:17.543666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f35480-2db2-4f0c-95ad-6c34f7ae4b08 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:17.544199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f35480-2db2-4f0c-95ad-6c34f7ae4b08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:17.545924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f35480-2db2-4f0c-95ad-6c34f7ae4b08 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:17.553613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 421/838] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jul 19 20:41:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:17.562164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f35480-2db2-4f0c-95ad-6c34f7ae4b08 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:17.588490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de6383-7586-431c-9cb9-c14e3519ab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:17.589747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de6383-7586-431c-9cb9-c14e3519ab91 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:17.590261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de6383-7586-431c-9cb9-c14e3519ab91 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:17.590801 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de6383-7586-431c-9cb9-c14e3519ab91 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:17.591805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de6383-7586-431c-9cb9-c14e3519ab91 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:17.593843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de6383-7586-431c-9cb9-c14e3519ab91 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-97de6383-7586-431c-9cb9-c14e3519ab91 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:17.609956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:17.611075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:41:17.611923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:17.615076 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 418/839] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:41:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:17.620976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de6383-7586-431c-9cb9-c14e3519ab91 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:17.719934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f35480-2db2-4f0c-95ad-6c34f7ae4b08 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:17.777896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662ab1d-8ab3-480c-86ca-d3350400624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:17.779534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662ab1d-8ab3-480c-86ca-d3350400624f None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:17.779823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662ab1d-8ab3-480c-86ca-d3350400624f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:17.780100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662ab1d-8ab3-480c-86ca-d3350400624f None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/812342d822f54aaf98e324d5d4e56753/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:17.785297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662ab1d-8ab3-480c-86ca-d3350400624f None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=812342d822f54aaf98e324d5d4e56753, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:17.788315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662ab1d-8ab3-480c-86ca-d3350400624f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:17.790302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:17.805828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de6383-7586-431c-9cb9-c14e3519ab91 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:17.807450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 422/840] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:41:17 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/812342d822f54aaf98e324d5d4e56753/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:41:17.856345 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35876a3c-3eee-4d62-9ec7-9915974e9cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:17.857059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35876a3c-3eee-4d62-9ec7-9915974e9cf9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:17.857340 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35876a3c-3eee-4d62-9ec7-9915974e9cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:17.857613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35876a3c-3eee-4d62-9ec7-9915974e9cf9 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:17.858356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35876a3c-3eee-4d62-9ec7-9915974e9cf9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:17.858662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35876a3c-3eee-4d62-9ec7-9915974e9cf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:17.860166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35876a3c-3eee-4d62-9ec7-9915974e9cf9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:17.866356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 419/841] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:41:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:17.871931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662ab1d-8ab3-480c-86ca-d3350400624f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:17.911589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:18.011561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662ab1d-8ab3-480c-86ca-d3350400624f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:18.059052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1721bd-10e9-4c2e-880f-7a39b31760c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:18.061470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1721bd-10e9-4c2e-880f-7a39b31760c3 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:18.061773 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1721bd-10e9-4c2e-880f-7a39b31760c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:18.062406 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1721bd-10e9-4c2e-880f-7a39b31760c3 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:18.064146 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1721bd-10e9-4c2e-880f-7a39b31760c3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:18.064982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1721bd-10e9-4c2e-880f-7a39b31760c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:18.070490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1721bd-10e9-4c2e-880f-7a39b31760c3 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:18.080751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 423/842] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:41:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:18.706898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185aad1-f4a6-4f75-acc3-df2769a8aea1 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:18.707456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185aad1-f4a6-4f75-acc3-df2769a8aea1 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:18.707456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185aad1-f4a6-4f75-acc3-df2769a8aea1 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:18.709264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 420/843] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:18.715248 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c12ef-7f9b-4490-9b5e-911a816ce83f None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:18.715549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c12ef-7f9b-4490-9b5e-911a816ce83f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:18.715818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c12ef-7f9b-4490-9b5e-911a816ce83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:18.733570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-d15c12ef-7f9b-4490-9b5e-911a816ce83f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:18.734785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d15c12ef-7f9b-4490-9b5e-911a816ce83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:18.808744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 424/844] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:18.834079 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185aad1-f4a6-4f75-acc3-df2769a8aea1 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:18.963157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185aad1-f4a6-4f75-acc3-df2769a8aea1 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:19.000354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e7a18-9ac3-4433-ab5f-aeb91e96c465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:19.001537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e7a18-9ac3-4433-ab5f-aeb91e96c465 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:19.002047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e7a18-9ac3-4433-ab5f-aeb91e96c465 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:19.002535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e7a18-9ac3-4433-ab5f-aeb91e96c465 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:19.014523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e7a18-9ac3-4433-ab5f-aeb91e96c465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-8b3e7a18-9ac3-4433-ab5f-aeb91e96c465 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:19.023502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:19.024512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:41:19.025382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:19.032431 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 421/845] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:41:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:19.034543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c12ef-7f9b-4490-9b5e-911a816ce83f None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:19.064592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:19.150835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c12ef-7f9b-4490-9b5e-911a816ce83f None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:19.186644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c44be3-5808-444d-b2bb-88deeb6e612a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:19.187324 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c44be3-5808-444d-b2bb-88deeb6e612a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:19.187610 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c44be3-5808-444d-b2bb-88deeb6e612a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:19.187889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c44be3-5808-444d-b2bb-88deeb6e612a None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:19.188758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c44be3-5808-444d-b2bb-88deeb6e612a None admin] RBAC: Adding query filter params (name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:19.189072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c44be3-5808-444d-b2bb-88deeb6e612a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:19.190454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c44be3-5808-444d-b2bb-88deeb6e612a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:19.199853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 425/846] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:41:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:19.217539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e7a18-9ac3-4433-ab5f-aeb91e96c465 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:19.240806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:19.337957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e7a18-9ac3-4433-ab5f-aeb91e96c465 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:19.376055 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4082efd-dcd3-475f-8c36-bad80e4c83b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:19.377295 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4082efd-dcd3-475f-8c36-bad80e4c83b7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:19.377796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4082efd-dcd3-475f-8c36-bad80e4c83b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:19.378285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4082efd-dcd3-475f-8c36-bad80e4c83b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:19.390533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4082efd-dcd3-475f-8c36-bad80e4c83b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:19.392314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4082efd-dcd3-475f-8c36-bad80e4c83b7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-c4082efd-dcd3-475f-8c36-bad80e4c83b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:19.411536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:19.412557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:19.413510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:19.413510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:19.413510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:19.413510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:19.420502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 422/847] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:19.422901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c44be3-5808-444d-b2bb-88deeb6e612a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:19.527411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:19.551830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c44be3-5808-444d-b2bb-88deeb6e612a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:19.589537 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8978-d6fc-4a2b-9180-797afbfa3b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:19.590291 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8978-d6fc-4a2b-9180-797afbfa3b7b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:19.590582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8978-d6fc-4a2b-9180-797afbfa3b7b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:19.590868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8978-d6fc-4a2b-9180-797afbfa3b7b None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:19.591601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b8978-d6fc-4a2b-9180-797afbfa3b7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:19.591984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b8978-d6fc-4a2b-9180-797afbfa3b7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:19.593472 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b8978-d6fc-4a2b-9180-797afbfa3b7b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:19.607370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 426/848] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:19.614351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4082efd-dcd3-475f-8c36-bad80e4c83b7 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:19.720046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:19.764600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4082efd-dcd3-475f-8c36-bad80e4c83b7 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:19.810274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bf4a0-ee12-4897-ab6a-3569213111a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:19.811144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bf4a0-ee12-4897-ab6a-3569213111a4 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:19.811426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bf4a0-ee12-4897-ab6a-3569213111a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:19.811702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bf4a0-ee12-4897-ab6a-3569213111a4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:19.827082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bf4a0-ee12-4897-ab6a-3569213111a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:19.827082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bf4a0-ee12-4897-ab6a-3569213111a4 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-348bf4a0-ee12-4897-ab6a-3569213111a4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:19.840438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:19.841496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:19.842534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:19.842534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:19.842534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:19.842534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:19.842534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:19.842534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:19.842534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:19.842534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:19.842534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:19.842534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:19.848465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 423/849] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jul 19 20:41:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:19.851484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8978-d6fc-4a2b-9180-797afbfa3b7b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:19.922452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c16a2-7e59-47d2-a2bc-f34b29c752ee None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:19.922967 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c16a2-7e59-47d2-a2bc-f34b29c752ee None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:19.922967 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c16a2-7e59-47d2-a2bc-f34b29c752ee None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:19.924960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 424/850] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:19.932777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226848f-6672-47ff-b85a-a86875d02559 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:19.933108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226848f-6672-47ff-b85a-a86875d02559 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:19.933336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226848f-6672-47ff-b85a-a86875d02559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:19.951601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-9226848f-6672-47ff-b85a-a86875d02559 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:19.953365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-9226848f-6672-47ff-b85a-a86875d02559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:19.998968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8978-d6fc-4a2b-9180-797afbfa3b7b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:20.040475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 425/851] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:20.048793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663e95c-4fb7-4af4-acd9-3ee9cbb0f4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:20.050231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663e95c-4fb7-4af4-acd9-3ee9cbb0f4a0 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:20.050763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663e95c-4fb7-4af4-acd9-3ee9cbb0f4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:20.051441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663e95c-4fb7-4af4-acd9-3ee9cbb0f4a0 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.052567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663e95c-4fb7-4af4-acd9-3ee9cbb0f4a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:20.053223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663e95c-4fb7-4af4-acd9-3ee9cbb0f4a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:20.054996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663e95c-4fb7-4af4-acd9-3ee9cbb0f4a0 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:20.064063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226848f-6672-47ff-b85a-a86875d02559 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:20.071946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:20.092065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 427/852] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jul 19 20:41:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:20.093878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663e95c-4fb7-4af4-acd9-3ee9cbb0f4a0 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:20.157192 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:20.227974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226848f-6672-47ff-b85a-a86875d02559 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:20.235742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663e95c-4fb7-4af4-acd9-3ee9cbb0f4a0 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:20.277505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab6d80-7ac0-4a2e-a570-cf8242f61ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:20.278207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab6d80-7ac0-4a2e-a570-cf8242f61ff4 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:20.278543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab6d80-7ac0-4a2e-a570-cf8242f61ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:20.278905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab6d80-7ac0-4a2e-a570-cf8242f61ff4 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.279796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab6d80-7ac0-4a2e-a570-cf8242f61ff4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:20.281587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab6d80-7ac0-4a2e-a570-cf8242f61ff4 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:20.291949 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7756c-bbd6-49f2-a85d-a59a48d86174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:20.293037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7756c-bbd6-49f2-a85d-a59a48d86174 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:20.293310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7756c-bbd6-49f2-a85d-a59a48d86174 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:20.293746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7756c-bbd6-49f2-a85d-a59a48d86174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.302233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 426/853] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:41:20 2022] POST /identity/v3/services => generated 269 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:20.302233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a552fa-31d3-4856-9747-b5d09004fcab None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:20.302233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a552fa-31d3-4856-9747-b5d09004fcab None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:20.302233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a552fa-31d3-4856-9747-b5d09004fcab None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.302233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 427/854] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:20.309761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20538cf0-1114-4b56-93e5-c52f6699677f None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:20.310137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20538cf0-1114-4b56-93e5-c52f6699677f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:20.310424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20538cf0-1114-4b56-93e5-c52f6699677f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.336230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-20538cf0-1114-4b56-93e5-c52f6699677f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:20.337782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-20538cf0-1114-4b56-93e5-c52f6699677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:20.404816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:20.409018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7756c-bbd6-49f2-a85d-a59a48d86174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:20.411904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7756c-bbd6-49f2-a85d-a59a48d86174 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:20.460799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 428/855] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:20 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:20.489269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20538cf0-1114-4b56-93e5-c52f6699677f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:20.560364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 428/856] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:20 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:41:20.562449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7756c-bbd6-49f2-a85d-a59a48d86174 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:20.584688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:20.639736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20538cf0-1114-4b56-93e5-c52f6699677f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:20.700826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4510d8-24a9-4ae4-a7c0-79038cbff02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:20.701625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4510d8-24a9-4ae4-a7c0-79038cbff02a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:20.702015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4510d8-24a9-4ae4-a7c0-79038cbff02a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:20.702195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4510d8-24a9-4ae4-a7c0-79038cbff02a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.709307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:20.716647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7756c-bbd6-49f2-a85d-a59a48d86174 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:20.720899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4510d8-24a9-4ae4-a7c0-79038cbff02a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-cc4510d8-24a9-4ae4-a7c0-79038cbff02a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:20.733003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:20.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:20.734847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:20.735518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 429/857] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:41:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:20.737336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9515-de76-458b-a33e-3727bd2de8e8 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:20.737824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9515-de76-458b-a33e-3727bd2de8e8 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:20.738291 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9515-de76-458b-a33e-3727bd2de8e8 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.741397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 430/858] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:20.741767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa9515-de76-458b-a33e-3727bd2de8e8 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:20.762860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10cddb-9e8f-409c-971b-5271b58ea742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:20.763755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cddb-9e8f-409c-971b-5271b58ea742 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:20.763923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cddb-9e8f-409c-971b-5271b58ea742 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:20.764202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cddb-9e8f-409c-971b-5271b58ea742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.775733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10cddb-9e8f-409c-971b-5271b58ea742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:20.777347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10cddb-9e8f-409c-971b-5271b58ea742 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-ef10cddb-9e8f-409c-971b-5271b58ea742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:20.796601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:20.797576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:20.798532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:20.798532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:20.798532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:20.798532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:20.804152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 429/859] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:20.805611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9f011-539a-4ab4-8e31-e649038d6873 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:20.805896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9f011-539a-4ab4-8e31-e649038d6873 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:20.806149 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9f011-539a-4ab4-8e31-e649038d6873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.826420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c0b9f011-539a-4ab4-8e31-e649038d6873 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:20.827669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c0b9f011-539a-4ab4-8e31-e649038d6873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:20.897656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa9515-de76-458b-a33e-3727bd2de8e8 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:20.908376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:20.916413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 430/860] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:20 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:20.916413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9f011-539a-4ab4-8e31-e649038d6873 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:20.954903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127d142-b45d-49c8-8216-dba99c5d4c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:20.955576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127d142-b45d-49c8-8216-dba99c5d4c0f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:20.955861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127d142-b45d-49c8-8216-dba99c5d4c0f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:20.956129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127d142-b45d-49c8-8216-dba99c5d4c0f None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:20.956961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127d142-b45d-49c8-8216-dba99c5d4c0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:20.957755 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127d142-b45d-49c8-8216-dba99c5d4c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:20.959128 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127d142-b45d-49c8-8216-dba99c5d4c0f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:20.973106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 431/861] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:20.973106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127d142-b45d-49c8-8216-dba99c5d4c0f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:21.037028 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:21.069639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9f011-539a-4ab4-8e31-e649038d6873 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:21.106981 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127d142-b45d-49c8-8216-dba99c5d4c0f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:21.128083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408edb6f-df36-4017-a128-beb255537363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:21.129740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408edb6f-df36-4017-a128-beb255537363 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:21.130211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408edb6f-df36-4017-a128-beb255537363 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:21.130667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408edb6f-df36-4017-a128-beb255537363 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:21.131937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408edb6f-df36-4017-a128-beb255537363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:21.132487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408edb6f-df36-4017-a128-beb255537363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:21.134905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408edb6f-df36-4017-a128-beb255537363 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:21.150713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 431/862] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:21.151706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408edb6f-df36-4017-a128-beb255537363 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:21.162594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852abb8-92f4-4de8-8d52-eb7a2e518b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:21.163593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852abb8-92f4-4de8-8d52-eb7a2e518b96 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:21.164064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852abb8-92f4-4de8-8d52-eb7a2e518b96 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:21.164560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852abb8-92f4-4de8-8d52-eb7a2e518b96 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:21.165528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852abb8-92f4-4de8-8d52-eb7a2e518b96 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:21.167006 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852abb8-92f4-4de8-8d52-eb7a2e518b96 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:21.192354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 432/863] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:41:20 2022] POST /identity/v3/services => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:21.192354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852abb8-92f4-4de8-8d52-eb7a2e518b96 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:21.249396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:21.292178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408edb6f-df36-4017-a128-beb255537363 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:21.347263 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010696fa-91c8-453d-b05b-a7d64a1e0d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:21.348892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010696fa-91c8-453d-b05b-a7d64a1e0d44 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:21.349236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010696fa-91c8-453d-b05b-a7d64a1e0d44 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:21.349594 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010696fa-91c8-453d-b05b-a7d64a1e0d44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:21.354410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:21.365837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010696fa-91c8-453d-b05b-a7d64a1e0d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:21.367432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010696fa-91c8-453d-b05b-a7d64a1e0d44 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:21.372400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852abb8-92f4-4de8-8d52-eb7a2e518b96 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-010696fa-91c8-453d-b05b-a7d64a1e0d44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:21.386987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:21.387934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:21.388888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:21.388888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:21.388888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:21.388888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:21.392374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 432/864] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:21.396208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010696fa-91c8-453d-b05b-a7d64a1e0d44 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:21.417283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda6361-96bd-46d1-9e44-38a0f4033fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:21.418132 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda6361-96bd-46d1-9e44-38a0f4033fda None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:21.418418 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda6361-96bd-46d1-9e44-38a0f4033fda None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:21.418692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda6361-96bd-46d1-9e44-38a0f4033fda None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:21.429776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda6361-96bd-46d1-9e44-38a0f4033fda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:21.431250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda6361-96bd-46d1-9e44-38a0f4033fda None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-1cda6361-96bd-46d1-9e44-38a0f4033fda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:21.447437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:21.448790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:21.449772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:21.449772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:21.449772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:21.449772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:21.449772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:21.449772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:21.449772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:21.449772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:21.449772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 433/865] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:21.454850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda6361-96bd-46d1-9e44-38a0f4033fda None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:21.530582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010696fa-91c8-453d-b05b-a7d64a1e0d44 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:21.571211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8ed4f-913a-4b68-b438-4e9818e43f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:21.571894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8ed4f-913a-4b68-b438-4e9818e43f09 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:21.572181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8ed4f-913a-4b68-b438-4e9818e43f09 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:21.572581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8ed4f-913a-4b68-b438-4e9818e43f09 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:21.573316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8ed4f-913a-4b68-b438-4e9818e43f09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:21.573627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8ed4f-913a-4b68-b438-4e9818e43f09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:21.575070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8ed4f-913a-4b68-b438-4e9818e43f09 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:21.580947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:21.589546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda6361-96bd-46d1-9e44-38a0f4033fda None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:21.597035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 433/866] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:21.601693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8ed4f-913a-4b68-b438-4e9818e43f09 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:21.642305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7961c7-cfb3-4e5a-89d7-177df6a7a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:21.643414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7961c7-cfb3-4e5a-89d7-177df6a7a743 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:21.643921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7961c7-cfb3-4e5a-89d7-177df6a7a743 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:21.645066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7961c7-cfb3-4e5a-89d7-177df6a7a743 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:21.646255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7961c7-cfb3-4e5a-89d7-177df6a7a743 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:21.646576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7961c7-cfb3-4e5a-89d7-177df6a7a743 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:21.648076 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7961c7-cfb3-4e5a-89d7-177df6a7a743 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:21.668585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 434/867] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:21.672102 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7961c7-cfb3-4e5a-89d7-177df6a7a743 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:21.683052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:21.767141 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8ed4f-913a-4b68-b438-4e9818e43f09 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:21.803954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7961c7-cfb3-4e5a-89d7-177df6a7a743 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:21.814042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d00b9-fd3d-40dd-8191-064077611546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:21.814992 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d00b9-fd3d-40dd-8191-064077611546 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:21.815369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d00b9-fd3d-40dd-8191-064077611546 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:21.815742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d00b9-fd3d-40dd-8191-064077611546 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:21.824365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d00b9-fd3d-40dd-8191-064077611546 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:21.826140 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d00b9-fd3d-40dd-8191-064077611546 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-951d00b9-fd3d-40dd-8191-064077611546 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:21.843179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:21.844303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:21.845330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:21.845330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:21.845330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:21.845330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:21.845330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:21.845330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:21.845330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:21.845330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:21.845330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 20:41:21.845330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:21.848465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 434/868] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:41:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:21.854361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d00b9-fd3d-40dd-8191-064077611546 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:21.854989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe42994-e2d6-4827-9f14-b6fe1afdbbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:21.856038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe42994-e2d6-4827-9f14-b6fe1afdbbaa None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:21.856568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe42994-e2d6-4827-9f14-b6fe1afdbbaa None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:21.857107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe42994-e2d6-4827-9f14-b6fe1afdbbaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:21.858262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe42994-e2d6-4827-9f14-b6fe1afdbbaa None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=e2f34a304e714f838d68dfdf214e4712) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:21.858790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe42994-e2d6-4827-9f14-b6fe1afdbbaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:21.860612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe42994-e2d6-4827-9f14-b6fe1afdbbaa None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:21.867929 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 435/869] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:21 2022] GET /identity/v3/role_assignments?user.id=e2f34a304e714f838d68dfdf214e4712&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:22.030544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d00b9-fd3d-40dd-8191-064077611546 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:22.041022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:22.071330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee015df-a6b1-4000-b21a-7799e63c87d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:22.072098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee015df-a6b1-4000-b21a-7799e63c87d4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:22.072289 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee015df-a6b1-4000-b21a-7799e63c87d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:22.072580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee015df-a6b1-4000-b21a-7799e63c87d4 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:22.073434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee015df-a6b1-4000-b21a-7799e63c87d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:22.073731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee015df-a6b1-4000-b21a-7799e63c87d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:22.075164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee015df-a6b1-4000-b21a-7799e63c87d4 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:22.095462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 435/870] 10.222.0.85 () {56 vars in 1135 bytes} [Tue Jul 19 20:41:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:22.101208 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe42994-e2d6-4827-9f14-b6fe1afdbbaa None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:22.118197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:22.232085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe42994-e2d6-4827-9f14-b6fe1afdbbaa None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:22.270372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627d399-bf63-4637-9e75-0fd6445b488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:22.271948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627d399-bf63-4637-9e75-0fd6445b488c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:22.272697 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627d399-bf63-4637-9e75-0fd6445b488c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:22.273401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627d399-bf63-4637-9e75-0fd6445b488c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:22.371326 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627d399-bf63-4637-9e75-0fd6445b488c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:22.374203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627d399-bf63-4637-9e75-0fd6445b488c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:22.397550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:22.501544 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 436/871] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:22 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:41:22.507245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee015df-a6b1-4000-b21a-7799e63c87d4 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:22.627589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee015df-a6b1-4000-b21a-7799e63c87d4 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:22.667795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66935e8c-d036-4b2f-96d5-db5a325b6f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:22.669075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66935e8c-d036-4b2f-96d5-db5a325b6f97 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:22.669559 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66935e8c-d036-4b2f-96d5-db5a325b6f97 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:22.670040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66935e8c-d036-4b2f-96d5-db5a325b6f97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:22.685467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66935e8c-d036-4b2f-96d5-db5a325b6f97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:22.687384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66935e8c-d036-4b2f-96d5-db5a325b6f97 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-66935e8c-d036-4b2f-96d5-db5a325b6f97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:22.709241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:22.710200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:22.711093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:22.711093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:22.711093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:22.711093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:22.713521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 436/872] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:22.719620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627d399-bf63-4637-9e75-0fd6445b488c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:22.747462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc00527-f9b5-4a11-b1bf-6c2ce046c22e None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:22.747826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc00527-f9b5-4a11-b1bf-6c2ce046c22e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:22.748149 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc00527-f9b5-4a11-b1bf-6c2ce046c22e None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:22.755322 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 437/873] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:22.758352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3a2a-4a04-4d2d-9dc1-4199251bec39 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:22.758844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3a2a-4a04-4d2d-9dc1-4199251bec39 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:22.759319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3a2a-4a04-4d2d-9dc1-4199251bec39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:22.785584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c66d3a2a-4a04-4d2d-9dc1-4199251bec39 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:22.787488 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c66d3a2a-4a04-4d2d-9dc1-4199251bec39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:22.811734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:22.872202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627d399-bf63-4637-9e75-0fd6445b488c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:22.884448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 438/874] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:22.894271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:22.906882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d3a2a-4a04-4d2d-9dc1-4199251bec39 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:22.928773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f1392-0569-4bbc-8a6c-6dfbf7c140fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:22.929891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f1392-0569-4bbc-8a6c-6dfbf7c140fc None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:22.930493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f1392-0569-4bbc-8a6c-6dfbf7c140fc None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:22.930993 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f1392-0569-4bbc-8a6c-6dfbf7c140fc None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:22.932273 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f1392-0569-4bbc-8a6c-6dfbf7c140fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:22.932946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f1392-0569-4bbc-8a6c-6dfbf7c140fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:22.934916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f1392-0569-4bbc-8a6c-6dfbf7c140fc None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:22.950851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 437/875] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:22.960884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f1392-0569-4bbc-8a6c-6dfbf7c140fc None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:23.043170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d3a2a-4a04-4d2d-9dc1-4199251bec39 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:23.090335 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730dc3-9fb4-4640-a1ee-fc25ce7f800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:23.091213 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730dc3-9fb4-4640-a1ee-fc25ce7f800c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.091565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730dc3-9fb4-4640-a1ee-fc25ce7f800c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.091715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730dc3-9fb4-4640-a1ee-fc25ce7f800c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.094956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1730dc3-9fb4-4640-a1ee-fc25ce7f800c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:23.097070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1730dc3-9fb4-4640-a1ee-fc25ce7f800c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:23.103355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f1392-0569-4bbc-8a6c-6dfbf7c140fc None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-e1730dc3-9fb4-4640-a1ee-fc25ce7f800c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:23.114170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:23.115337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:41:23.116288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:23.121005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 439/876] 10.222.0.85 () {56 vars in 1132 bytes} [Tue Jul 19 20:41:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:23.122202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730dc3-9fb4-4640-a1ee-fc25ce7f800c None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:23.140738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:23.154115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea220e-4898-4ad0-ba53-36bfb623f0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:23.155666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea220e-4898-4ad0-ba53-36bfb623f0bb None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.156208 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea220e-4898-4ad0-ba53-36bfb623f0bb None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.156704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea220e-4898-4ad0-ba53-36bfb623f0bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.167551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea220e-4898-4ad0-ba53-36bfb623f0bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:23.169729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea220e-4898-4ad0-ba53-36bfb623f0bb None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-7eea220e-4898-4ad0-ba53-36bfb623f0bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:23.188056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:23.189053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:23.189970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:23.189970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:23.189970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:23.189970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:23.189970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:23.189970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:23.189970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:23.189970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:23.192627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 438/877] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:23.198368 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea220e-4898-4ad0-ba53-36bfb623f0bb None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:23.251943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:23.278341 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730dc3-9fb4-4640-a1ee-fc25ce7f800c None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:23.334627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e60a6a-0c3c-41b9-805f-3988ef11d796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:23.336237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e60a6a-0c3c-41b9-805f-3988ef11d796 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.337001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e60a6a-0c3c-41b9-805f-3988ef11d796 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.337397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea220e-4898-4ad0-ba53-36bfb623f0bb None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:23.338036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e60a6a-0c3c-41b9-805f-3988ef11d796 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.339714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e60a6a-0c3c-41b9-805f-3988ef11d796 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:23.340622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e60a6a-0c3c-41b9-805f-3988ef11d796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:23.343783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e60a6a-0c3c-41b9-805f-3988ef11d796 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:23.354824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 440/878] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jul 19 20:41:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:23.363627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e60a6a-0c3c-41b9-805f-3988ef11d796 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:23.380215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7f608-285a-4356-a8e0-84f8b2c509aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:23.381111 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f608-285a-4356-a8e0-84f8b2c509aa None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.381458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f608-285a-4356-a8e0-84f8b2c509aa None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.381744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f608-285a-4356-a8e0-84f8b2c509aa None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.382554 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7f608-285a-4356-a8e0-84f8b2c509aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:23.382917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7f608-285a-4356-a8e0-84f8b2c509aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:23.384502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7f608-285a-4356-a8e0-84f8b2c509aa None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:23.401421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 439/879] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:23.413038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7f608-285a-4356-a8e0-84f8b2c509aa None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:23.478930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:23.507426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e60a6a-0c3c-41b9-805f-3988ef11d796 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:23.546581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2be68-54c6-44fe-9f1d-6c47a151ef21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:23.547644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2be68-54c6-44fe-9f1d-6c47a151ef21 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.548161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2be68-54c6-44fe-9f1d-6c47a151ef21 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.548702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2be68-54c6-44fe-9f1d-6c47a151ef21 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.549795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2be68-54c6-44fe-9f1d-6c47a151ef21 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:23.550325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2be68-54c6-44fe-9f1d-6c47a151ef21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:23.552620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2be68-54c6-44fe-9f1d-6c47a151ef21 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:23.559348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 441/880] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jul 19 20:41:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:23.569321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2be68-54c6-44fe-9f1d-6c47a151ef21 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:23.597251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:23.609534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7f608-285a-4356-a8e0-84f8b2c509aa None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:23.649056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961b900-207b-4983-b2cc-020c841d92fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:23.650221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961b900-207b-4983-b2cc-020c841d92fa None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.650707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961b900-207b-4983-b2cc-020c841d92fa None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.651255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961b900-207b-4983-b2cc-020c841d92fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.652525 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961b900-207b-4983-b2cc-020c841d92fa None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=812342d822f54aaf98e324d5d4e56753) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:23.653059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961b900-207b-4983-b2cc-020c841d92fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:23.654820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961b900-207b-4983-b2cc-020c841d92fa None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:23.668521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 440/881] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:23 2022] GET /identity/v3/role_assignments?user.id=812342d822f54aaf98e324d5d4e56753&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:23.766789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2be68-54c6-44fe-9f1d-6c47a151ef21 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:23.801248 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f645d47-6148-44db-8b8d-6fee28e60def None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.801710 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f645d47-6148-44db-8b8d-6fee28e60def None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.802181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f645d47-6148-44db-8b8d-6fee28e60def None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.805079 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 441/882] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:23.814422 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa8eec-5e6d-4cb9-bad7-8e5ab7930612 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.814927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa8eec-5e6d-4cb9-bad7-8e5ab7930612 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.815412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa8eec-5e6d-4cb9-bad7-8e5ab7930612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.823333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b775a-e234-48cb-98f3-a4aa38e28ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:23.824016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b775a-e234-48cb-98f3-a4aa38e28ab5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:23.824307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b775a-e234-48cb-98f3-a4aa38e28ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:23.824658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b775a-e234-48cb-98f3-a4aa38e28ab5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:23.825435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b775a-e234-48cb-98f3-a4aa38e28ab5 None admin] RBAC: Adding query filter params (interface=public, service_id=c4abc31a83594197902a068e9cec6b92, region_id=RegionOne) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:23.825747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b775a-e234-48cb-98f3-a4aa38e28ab5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:23.827159 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b775a-e234-48cb-98f3-a4aa38e28ab5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:23.833564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 442/883] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jul 19 20:41:23 2022] GET /identity/v3/endpoints?service_id=c4abc31a83594197902a068e9cec6b92&interface=public®ion_id=RegionOne => generated 194 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:23.841375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b775a-e234-48cb-98f3-a4aa38e28ab5 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:23.842175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-f5fa8eec-5e6d-4cb9-bad7-8e5ab7930612 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:23.843473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f5fa8eec-5e6d-4cb9-bad7-8e5ab7930612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:23.846863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:23.946492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 442/884] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:23.979833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa8eec-5e6d-4cb9-bad7-8e5ab7930612 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:24.004541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b775a-e234-48cb-98f3-a4aa38e28ab5 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:24.058964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726530e5-3492-44c9-ac47-999066393de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:24.060631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726530e5-3492-44c9-ac47-999066393de5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:24.061282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726530e5-3492-44c9-ac47-999066393de5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:24.061852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726530e5-3492-44c9-ac47-999066393de5 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:24.062883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726530e5-3492-44c9-ac47-999066393de5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:24.065087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726530e5-3492-44c9-ac47-999066393de5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:24.075578 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 443/885] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jul 19 20:41:23 2022] GET /identity/v3/services => generated 1268 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:41:24.128785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:24.133100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa8eec-5e6d-4cb9-bad7-8e5ab7930612 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:24.179486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621011e9-4608-4a63-83d2-50fbf67ac545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:24.180339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621011e9-4608-4a63-83d2-50fbf67ac545 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:24.180643 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621011e9-4608-4a63-83d2-50fbf67ac545 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:24.180921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621011e9-4608-4a63-83d2-50fbf67ac545 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:24.181769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621011e9-4608-4a63-83d2-50fbf67ac545 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:24.184242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621011e9-4608-4a63-83d2-50fbf67ac545 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-621011e9-4608-4a63-83d2-50fbf67ac545 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:24.198350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:24.199573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:24.200950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 443/886] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:41:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:24.207850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726530e5-3492-44c9-ac47-999066393de5 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:24.302351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:24.353407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726530e5-3492-44c9-ac47-999066393de5 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:24.397057 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83602608-f564-4dfe-acd3-c76a03efd8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:24.397868 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa842-e8d3-43b4-9e43-2f6d2d0ae39e None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:24.398156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa842-e8d3-43b4-9e43-2f6d2d0ae39e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:24.398438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa842-e8d3-43b4-9e43-2f6d2d0ae39e None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:24.398726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83602608-f564-4dfe-acd3-c76a03efd8d6 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:24.399290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83602608-f564-4dfe-acd3-c76a03efd8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:24.399765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83602608-f564-4dfe-acd3-c76a03efd8d6 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:24.400081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 444/887] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:24.401255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83602608-f564-4dfe-acd3-c76a03efd8d6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:24.401936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83602608-f564-4dfe-acd3-c76a03efd8d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:24.403567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83602608-f564-4dfe-acd3-c76a03efd8d6 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:24.405669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6fad7-f63f-4bfe-9451-771c29c000f1 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:24.405956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6fad7-f63f-4bfe-9451-771c29c000f1 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:24.406231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6fad7-f63f-4bfe-9451-771c29c000f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:24.410819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 444/888] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:41:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:24.420380 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83602608-f564-4dfe-acd3-c76a03efd8d6 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:24.429123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-bff6fad7-f63f-4bfe-9451-771c29c000f1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:24.430924 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bff6fad7-f63f-4bfe-9451-771c29c000f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:24.544481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 445/889] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:24.559017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83602608-f564-4dfe-acd3-c76a03efd8d6 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:24.564534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6fad7-f63f-4bfe-9451-771c29c000f1 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:24.609297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:24.618278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b9abe-0d59-44dc-8b1f-c1f6bfc6e811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:24.619656 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b9abe-0d59-44dc-8b1f-c1f6bfc6e811 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:24.620736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b9abe-0d59-44dc-8b1f-c1f6bfc6e811 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:24.624465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b9abe-0d59-44dc-8b1f-c1f6bfc6e811 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:24.626193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b9abe-0d59-44dc-8b1f-c1f6bfc6e811 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:24.627601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b9abe-0d59-44dc-8b1f-c1f6bfc6e811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:24.630694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b9abe-0d59-44dc-8b1f-c1f6bfc6e811 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:24.639590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 445/890] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:41:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:24.647424 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b9abe-0d59-44dc-8b1f-c1f6bfc6e811 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:24.682199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:24.708211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6fad7-f63f-4bfe-9451-771c29c000f1 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:24.765067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e80b62-c34a-4556-b7de-3369ef23dfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:24.765763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e80b62-c34a-4556-b7de-3369ef23dfa0 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:24.766053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e80b62-c34a-4556-b7de-3369ef23dfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:24.766330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e80b62-c34a-4556-b7de-3369ef23dfa0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:24.780388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e80b62-c34a-4556-b7de-3369ef23dfa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:24.787798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b9abe-0d59-44dc-8b1f-c1f6bfc6e811 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-d4e80b62-c34a-4556-b7de-3369ef23dfa0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:24.789209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:24.790230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:24.791130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:24.796119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 446/891] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jul 19 20:41:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:24.797021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e80b62-c34a-4556-b7de-3369ef23dfa0 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:24.839944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae08bf-2b44-4745-a19d-a27c3113fe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:24.841184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae08bf-2b44-4745-a19d-a27c3113fe09 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:24.841724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae08bf-2b44-4745-a19d-a27c3113fe09 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:24.842327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae08bf-2b44-4745-a19d-a27c3113fe09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:24.843614 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae08bf-2b44-4745-a19d-a27c3113fe09 None admin] RBAC: Adding query filter params (interface=public, service_id=09308035e15a44aa9113203d42e15ad2, region_id=RegionOne) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:24.844433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae08bf-2b44-4745-a19d-a27c3113fe09 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:24.846466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae08bf-2b44-4745-a19d-a27c3113fe09 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:24.859545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 446/892] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jul 19 20:41:24 2022] GET /identity/v3/endpoints?service_id=09308035e15a44aa9113203d42e15ad2&interface=public®ion_id=RegionOne => generated 194 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:24.860992 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae08bf-2b44-4745-a19d-a27c3113fe09 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:24.915062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e80b62-c34a-4556-b7de-3369ef23dfa0 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:24.932463 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:24.968985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbca3c-c8f8-4bf1-ab01-b1c58933fb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:24.969846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbca3c-c8f8-4bf1-ab01-b1c58933fb30 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:24.970542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbca3c-c8f8-4bf1-ab01-b1c58933fb30 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:24.970836 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbca3c-c8f8-4bf1-ab01-b1c58933fb30 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:24.971895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbca3c-c8f8-4bf1-ab01-b1c58933fb30 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:24.972203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbca3c-c8f8-4bf1-ab01-b1c58933fb30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:24.974217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbca3c-c8f8-4bf1-ab01-b1c58933fb30 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:24.985983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 447/893] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jul 19 20:41:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:24.995745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbca3c-c8f8-4bf1-ab01-b1c58933fb30 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:24.995937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae08bf-2b44-4745-a19d-a27c3113fe09 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:24.996043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:25.048165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33f87d-03cc-4a53-aaaf-4a1b22b05f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:25.048895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33f87d-03cc-4a53-aaaf-4a1b22b05f78 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:25.049580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33f87d-03cc-4a53-aaaf-4a1b22b05f78 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:25.049887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33f87d-03cc-4a53-aaaf-4a1b22b05f78 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:25.051209 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33f87d-03cc-4a53-aaaf-4a1b22b05f78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:25.053547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33f87d-03cc-4a53-aaaf-4a1b22b05f78 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:25.062079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 447/894] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jul 19 20:41:24 2022] GET /identity/v3/services => generated 1268 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:41:25.134449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbca3c-c8f8-4bf1-ab01-b1c58933fb30 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:25.177606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875d5c1-d49b-459e-83e6-561fd5584ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:25.178462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875d5c1-d49b-459e-83e6-561fd5584ba1 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:25.178771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875d5c1-d49b-459e-83e6-561fd5584ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:25.179074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875d5c1-d49b-459e-83e6-561fd5584ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:25.194855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875d5c1-d49b-459e-83e6-561fd5584ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:25.196636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875d5c1-d49b-459e-83e6-561fd5584ba1 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-a875d5c1-d49b-459e-83e6-561fd5584ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:25.219885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:25.220917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:25.221867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:25.221867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:25.221867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:25.221867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:25.221867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 448/895] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:25.228944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33f87d-03cc-4a53-aaaf-4a1b22b05f78 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:25.375895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33f87d-03cc-4a53-aaaf-4a1b22b05f78 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:25.413274 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:25.427734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504145ad-fadb-4534-86fd-ef792976f85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:25.428442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504145ad-fadb-4534-86fd-ef792976f85c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:25.428738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504145ad-fadb-4534-86fd-ef792976f85c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:25.429041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504145ad-fadb-4534-86fd-ef792976f85c None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:25.429909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504145ad-fadb-4534-86fd-ef792976f85c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:25.430230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504145ad-fadb-4534-86fd-ef792976f85c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:25.432080 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504145ad-fadb-4534-86fd-ef792976f85c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:25.449318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 448/896] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:25.459078 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875d5c1-d49b-459e-83e6-561fd5584ba1 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:25.568704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:25.617182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875d5c1-d49b-459e-83e6-561fd5584ba1 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:25.659471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b6d3e-8f8f-4f0e-94d2-63ff2f730a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:25.660254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b6d3e-8f8f-4f0e-94d2-63ff2f730a77 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:25.660575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b6d3e-8f8f-4f0e-94d2-63ff2f730a77 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:25.660854 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b6d3e-8f8f-4f0e-94d2-63ff2f730a77 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:25.670367 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b6d3e-8f8f-4f0e-94d2-63ff2f730a77 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:25.671875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b6d3e-8f8f-4f0e-94d2-63ff2f730a77 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-f67b6d3e-8f8f-4f0e-94d2-63ff2f730a77 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:25.688357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:25.690640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:25.693503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:25.693503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:25.693503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:25.693503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:25.693503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:25.693503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:25.693503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:25.693503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:25.693503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:25.693503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:25.693503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 449/897] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jul 19 20:41:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:25.700360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504145ad-fadb-4534-86fd-ef792976f85c None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:25.805139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504145ad-fadb-4534-86fd-ef792976f85c None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:25.842965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b6d4e-2548-4dd7-8710-e90be423bb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:25.843647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b6d4e-2548-4dd7-8710-e90be423bb88 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:25.843936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b6d4e-2548-4dd7-8710-e90be423bb88 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:25.844200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b6d4e-2548-4dd7-8710-e90be423bb88 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:25.845100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b6d4e-2548-4dd7-8710-e90be423bb88 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:25.845454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b6d4e-2548-4dd7-8710-e90be423bb88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:25.846892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b6d4e-2548-4dd7-8710-e90be423bb88 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:25.866965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 449/898] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jul 19 20:41:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:25.872876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b6d3e-8f8f-4f0e-94d2-63ff2f730a77 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:25.996716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b6d3e-8f8f-4f0e-94d2-63ff2f730a77 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:26.015269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:26.039126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09793a-3743-4461-906d-c62ec4fc9c16 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.039467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09793a-3743-4461-906d-c62ec4fc9c16 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.039692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09793a-3743-4461-906d-c62ec4fc9c16 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.042252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 450/899] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:26.046001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1785c9-f4fa-4cae-b006-b4ea8c5eeaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:26.047028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1785c9-f4fa-4cae-b006-b4ea8c5eeaf7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.047298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e51aeb-6b44-46fe-8248-cc36874e1e24 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.047607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e51aeb-6b44-46fe-8248-cc36874e1e24 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.047886 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e51aeb-6b44-46fe-8248-cc36874e1e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.048289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1785c9-f4fa-4cae-b006-b4ea8c5eeaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.048800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1785c9-f4fa-4cae-b006-b4ea8c5eeaf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.065923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-57e51aeb-6b44-46fe-8248-cc36874e1e24 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:26.067881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-57e51aeb-6b44-46fe-8248-cc36874e1e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:26.120574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:26.149364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1785c9-f4fa-4cae-b006-b4ea8c5eeaf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:26.151012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1785c9-f4fa-4cae-b006-b4ea8c5eeaf7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:26.172709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 451/900] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:26.200161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e51aeb-6b44-46fe-8248-cc36874e1e24 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:26.276519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 450/901] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:25 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:41:26.284731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1785c9-f4fa-4cae-b006-b4ea8c5eeaf7 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:26.301489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:26.333624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e51aeb-6b44-46fe-8248-cc36874e1e24 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:26.371209 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77843a-e7c8-48e1-95e3-c1c653c517b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:26.372489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77843a-e7c8-48e1-95e3-c1c653c517b3 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.372976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77843a-e7c8-48e1-95e3-c1c653c517b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.373462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77843a-e7c8-48e1-95e3-c1c653c517b3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.374322 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77843a-e7c8-48e1-95e3-c1c653c517b3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:26.376254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77843a-e7c8-48e1-95e3-c1c653c517b3 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-3e77843a-e7c8-48e1-95e3-c1c653c517b3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:26.391267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:26.393216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:26.395381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:26.404507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 452/902] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jul 19 20:41:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:26.411403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77843a-e7c8-48e1-95e3-c1c653c517b3 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:26.437183 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1785c9-f4fa-4cae-b006-b4ea8c5eeaf7 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:26.457393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:26.482715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5c389-4fde-422b-a5a8-58b5a10a9ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:26.483663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5c389-4fde-422b-a5a8-58b5a10a9ff6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.484003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5c389-4fde-422b-a5a8-58b5a10a9ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.484370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5c389-4fde-422b-a5a8-58b5a10a9ff6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.496820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5c389-4fde-422b-a5a8-58b5a10a9ff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:26.498426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5c389-4fde-422b-a5a8-58b5a10a9ff6 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-59e5c389-4fde-422b-a5a8-58b5a10a9ff6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:26.519148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:26.520155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:26.525946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:26.525946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:26.525946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:26.525946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:26.525946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 451/903] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:26.525946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cae61-009f-4382-870a-6ef63b0e9c0c None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.525946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cae61-009f-4382-870a-6ef63b0e9c0c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.525946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cae61-009f-4382-870a-6ef63b0e9c0c None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.525946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 452/904] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:26.528987 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cae61-009f-4382-870a-6ef63b0e9c0c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:26.544211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77843a-e7c8-48e1-95e3-c1c653c517b3 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:26.583344 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c601654-230d-404f-ad90-1b8d7e045a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:26.584023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c601654-230d-404f-ad90-1b8d7e045a91 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.584306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c601654-230d-404f-ad90-1b8d7e045a91 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.584617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c601654-230d-404f-ad90-1b8d7e045a91 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.585354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c601654-230d-404f-ad90-1b8d7e045a91 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:26.585643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c601654-230d-404f-ad90-1b8d7e045a91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:26.587074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c601654-230d-404f-ad90-1b8d7e045a91 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:26.596378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 453/905] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jul 19 20:41:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:26.596378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00587b53-ad1f-4a22-af66-297ec9beefd7 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.596378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00587b53-ad1f-4a22-af66-297ec9beefd7 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.596378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00587b53-ad1f-4a22-af66-297ec9beefd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.611789 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:26.614713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-00587b53-ad1f-4a22-af66-297ec9beefd7 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:26.616881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-00587b53-ad1f-4a22-af66-297ec9beefd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:26.660253 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cae61-009f-4382-870a-6ef63b0e9c0c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:26.708259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde09440-9aef-4e04-aee5-f06bdd3db878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:26.708910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde09440-9aef-4e04-aee5-f06bdd3db878 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.709196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde09440-9aef-4e04-aee5-f06bdd3db878 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.709499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde09440-9aef-4e04-aee5-f06bdd3db878 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.710226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde09440-9aef-4e04-aee5-f06bdd3db878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:26.710540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde09440-9aef-4e04-aee5-f06bdd3db878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:26.712182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde09440-9aef-4e04-aee5-f06bdd3db878 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:26.723550 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 454/906] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:26.724452 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00587b53-ad1f-4a22-af66-297ec9beefd7 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:26.725371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 453/907] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:26.731108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde09440-9aef-4e04-aee5-f06bdd3db878 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:26.778014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:26.854895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde09440-9aef-4e04-aee5-f06bdd3db878 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:26.866306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00587b53-ad1f-4a22-af66-297ec9beefd7 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:26.886023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:26.896591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff00879-8d97-46d4-97e6-9fa92095b1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:26.897487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff00879-8d97-46d4-97e6-9fa92095b1f0 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.897776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff00879-8d97-46d4-97e6-9fa92095b1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.898054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff00879-8d97-46d4-97e6-9fa92095b1f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.905549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff00879-8d97-46d4-97e6-9fa92095b1f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:26.907017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff00879-8d97-46d4-97e6-9fa92095b1f0 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:26.909514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51e4ee-f679-49e1-b1fc-bd923a8988e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:26.911187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51e4ee-f679-49e1-b1fc-bd923a8988e9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:26.911677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51e4ee-f679-49e1-b1fc-bd923a8988e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:26.912220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51e4ee-f679-49e1-b1fc-bd923a8988e9 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:26.913130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51e4ee-f679-49e1-b1fc-bd923a8988e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:26.913591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51e4ee-f679-49e1-b1fc-bd923a8988e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:26.915136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51e4ee-f679-49e1-b1fc-bd923a8988e9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:26.921711 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 455/908] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jul 19 20:41:26 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:26.922555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51e4ee-f679-49e1-b1fc-bd923a8988e9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-5ff00879-8d97-46d4-97e6-9fa92095b1f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:26.926943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:26.927885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:26.929364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:26.929364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:26.929364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:26.929364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:26.929364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:26.929364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:26.929364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:26.929364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:26.933659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 454/909] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:26.934659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff00879-8d97-46d4-97e6-9fa92095b1f0 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:27.071011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:27.084262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51e4ee-f679-49e1-b1fc-bd923a8988e9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:27.090658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff00879-8d97-46d4-97e6-9fa92095b1f0 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:27.131533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26725788-fd45-4f1f-bd9b-47d6f36f12ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:27.132667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26725788-fd45-4f1f-bd9b-47d6f36f12ba None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:27.133139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26725788-fd45-4f1f-bd9b-47d6f36f12ba None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:27.133576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26725788-fd45-4f1f-bd9b-47d6f36f12ba None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:27.134374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26725788-fd45-4f1f-bd9b-47d6f36f12ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:27.135964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26725788-fd45-4f1f-bd9b-47d6f36f12ba None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:27.144979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be229b-cd5d-4ce5-af9a-f5c3ad321c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:27.146531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be229b-cd5d-4ce5-af9a-f5c3ad321c03 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:27.147090 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be229b-cd5d-4ce5-af9a-f5c3ad321c03 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:27.147589 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be229b-cd5d-4ce5-af9a-f5c3ad321c03 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:27.149728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be229b-cd5d-4ce5-af9a-f5c3ad321c03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:27.150311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be229b-cd5d-4ce5-af9a-f5c3ad321c03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-26725788-fd45-4f1f-bd9b-47d6f36f12ba None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:27.150580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:27.151574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:27.152532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 456/910] 10.222.0.85 () {56 vars in 1132 bytes} [Tue Jul 19 20:41:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:27.157505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be229b-cd5d-4ce5-af9a-f5c3ad321c03 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:27.158934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26725788-fd45-4f1f-bd9b-47d6f36f12ba None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:27.175601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 455/911] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:27.181093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be229b-cd5d-4ce5-af9a-f5c3ad321c03 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:27.251393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:27.287131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26725788-fd45-4f1f-bd9b-47d6f36f12ba None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:27.318960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be229b-cd5d-4ce5-af9a-f5c3ad321c03 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:27.327017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab2f09-9487-4c03-8562-404c77c8331d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:27.327938 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab2f09-9487-4c03-8562-404c77c8331d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:27.328403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab2f09-9487-4c03-8562-404c77c8331d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:27.328830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab2f09-9487-4c03-8562-404c77c8331d None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:27.329853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab2f09-9487-4c03-8562-404c77c8331d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:27.330413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab2f09-9487-4c03-8562-404c77c8331d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:27.332688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab2f09-9487-4c03-8562-404c77c8331d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:27.338946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 457/912] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jul 19 20:41:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:27.346843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab2f09-9487-4c03-8562-404c77c8331d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:27.375075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d45e5-0cdf-41e3-9c5a-a15a14ade29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:27.376046 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:27.376293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d45e5-0cdf-41e3-9c5a-a15a14ade29e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:27.376621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d45e5-0cdf-41e3-9c5a-a15a14ade29e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:27.377482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d45e5-0cdf-41e3-9c5a-a15a14ade29e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:27.378561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d45e5-0cdf-41e3-9c5a-a15a14ade29e None admin] RBAC: Adding query filter params (role.id=c22095e125674044bf3d2fe3122eeacd, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=e2f34a304e714f838d68dfdf214e4712) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:27.379077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d45e5-0cdf-41e3-9c5a-a15a14ade29e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:27.381132 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d45e5-0cdf-41e3-9c5a-a15a14ade29e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:27.388065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 456/913] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:27 2022] GET /identity/v3/role_assignments?user.id=e2f34a304e714f838d68dfdf214e4712&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=c22095e125674044bf3d2fe3122eeacd => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:27.389351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439655e-4056-491c-9cab-08164620075a None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:27.389645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439655e-4056-491c-9cab-08164620075a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:27.389930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439655e-4056-491c-9cab-08164620075a None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:27.392385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 457/914] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:27.402110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f8a00-f59f-4174-9bc9-7677eb657f3a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:27.402758 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f8a00-f59f-4174-9bc9-7677eb657f3a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:27.403386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f8a00-f59f-4174-9bc9-7677eb657f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:27.422767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4e2f8a00-f59f-4174-9bc9-7677eb657f3a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:27.424419 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4e2f8a00-f59f-4174-9bc9-7677eb657f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:27.495109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab2f09-9487-4c03-8562-404c77c8331d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:27.532501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 458/915] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:27.543221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3093df5-0581-46fc-a5c7-483962a6e08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:27.544442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3093df5-0581-46fc-a5c7-483962a6e08d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:27.545001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3093df5-0581-46fc-a5c7-483962a6e08d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:27.545513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3093df5-0581-46fc-a5c7-483962a6e08d None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:27.546520 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3093df5-0581-46fc-a5c7-483962a6e08d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:27.547043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3093df5-0581-46fc-a5c7-483962a6e08d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:27.548898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3093df5-0581-46fc-a5c7-483962a6e08d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:27.556061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f8a00-f59f-4174-9bc9-7677eb657f3a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:27.560860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 458/916] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jul 19 20:41:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:27.566887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3093df5-0581-46fc-a5c7-483962a6e08d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:27.611568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:27.682907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:27.706573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f8a00-f59f-4174-9bc9-7677eb657f3a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:27.711393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3093df5-0581-46fc-a5c7-483962a6e08d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:27.756400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d823044-2dc1-42a0-ae8d-8c90ed642e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:27.757617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d823044-2dc1-42a0-ae8d-8c90ed642e0e None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:27.757984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d823044-2dc1-42a0-ae8d-8c90ed642e0e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:27.758332 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d823044-2dc1-42a0-ae8d-8c90ed642e0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:27.759323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d823044-2dc1-42a0-ae8d-8c90ed642e0e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:27.761225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d823044-2dc1-42a0-ae8d-8c90ed642e0e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:27.766466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914813f0-80d9-4254-985d-30b1031cab42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:27.768107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914813f0-80d9-4254-985d-30b1031cab42 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:27.768931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914813f0-80d9-4254-985d-30b1031cab42 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:27.770300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914813f0-80d9-4254-985d-30b1031cab42 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:27.771842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914813f0-80d9-4254-985d-30b1031cab42 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:27.774870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914813f0-80d9-4254-985d-30b1031cab42 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:27.784613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 459/917] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jul 19 20:41:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-914813f0-80d9-4254-985d-30b1031cab42 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:27.806508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:27.807917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 20:41:27.808837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:27.815738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 459/918] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:41:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:27.817738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d823044-2dc1-42a0-ae8d-8c90ed642e0e None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:27.959129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d823044-2dc1-42a0-ae8d-8c90ed642e0e None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:28.001853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008c007-1c9e-44aa-aebe-d5e6c534432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:28.002930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008c007-1c9e-44aa-aebe-d5e6c534432c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:28.003218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008c007-1c9e-44aa-aebe-d5e6c534432c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:28.003544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008c007-1c9e-44aa-aebe-d5e6c534432c None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:28.004787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008c007-1c9e-44aa-aebe-d5e6c534432c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:28.005096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008c007-1c9e-44aa-aebe-d5e6c534432c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:28.006475 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008c007-1c9e-44aa-aebe-d5e6c534432c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:28.012390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 460/919] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:41:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:28.017961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914813f0-80d9-4254-985d-30b1031cab42 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:28.139525 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914813f0-80d9-4254-985d-30b1031cab42 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:28.142177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:28.177047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ba4e3-e575-48c9-95ac-9034590e6330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:28.177830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ba4e3-e575-48c9-95ac-9034590e6330 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:28.178358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ba4e3-e575-48c9-95ac-9034590e6330 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:28.179244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ba4e3-e575-48c9-95ac-9034590e6330 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:28.179961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ba4e3-e575-48c9-95ac-9034590e6330 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:28.180277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ba4e3-e575-48c9-95ac-9034590e6330 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:28.181712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ba4e3-e575-48c9-95ac-9034590e6330 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:28.187801 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 460/920] 10.222.0.85 () {56 vars in 1114 bytes} [Tue Jul 19 20:41:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:28.194257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008c007-1c9e-44aa-aebe-d5e6c534432c None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:28.208964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:28.326043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008c007-1c9e-44aa-aebe-d5e6c534432c None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:28.365529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065c750-fe5d-4689-8c27-5610953a5498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:28.366397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065c750-fe5d-4689-8c27-5610953a5498 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:28.366613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065c750-fe5d-4689-8c27-5610953a5498 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:28.366842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065c750-fe5d-4689-8c27-5610953a5498 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:28.367709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065c750-fe5d-4689-8c27-5610953a5498 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:28.369148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065c750-fe5d-4689-8c27-5610953a5498 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:28.392915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 461/921] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jul 19 20:41:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:29.356996 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd49c4-19e1-4843-bb8e-d2c40e33c400 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:29.357514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd49c4-19e1-4843-bb8e-d2c40e33c400 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:29.357514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd49c4-19e1-4843-bb8e-d2c40e33c400 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:29.359122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 461/922] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:29.365669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29defa35-73fc-4ae4-93b4-83f128eb6c2e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:29.366133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29defa35-73fc-4ae4-93b4-83f128eb6c2e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:29.366228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29defa35-73fc-4ae4-93b4-83f128eb6c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:29.384009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-29defa35-73fc-4ae4-93b4-83f128eb6c2e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:29.385239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-29defa35-73fc-4ae4-93b4-83f128eb6c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:29.452351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:29.478648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 462/923] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:29.501327 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd49c4-19e1-4843-bb8e-d2c40e33c400 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:29.638911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd49c4-19e1-4843-bb8e-d2c40e33c400 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:29.679684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93043d-5d78-4d76-9df9-efd9814c4142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:29.680443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93043d-5d78-4d76-9df9-efd9814c4142 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:29.680733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93043d-5d78-4d76-9df9-efd9814c4142 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:29.681015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93043d-5d78-4d76-9df9-efd9814c4142 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:29.681760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93043d-5d78-4d76-9df9-efd9814c4142 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:29.683469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93043d-5d78-4d76-9df9-efd9814c4142 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:29.700836 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 462/924] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:41:29 2022] POST /identity/v3/services => generated 241 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:29.729369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dbfd4-4aa3-47f1-bc0e-f21ce776e89a None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:29.729744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dbfd4-4aa3-47f1-bc0e-f21ce776e89a None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:29.730177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dbfd4-4aa3-47f1-bc0e-f21ce776e89a None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:29.733141 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 463/925] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:29.740978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20efeb2-5a05-48a2-aef7-692dba67373f None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:29.741334 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20efeb2-5a05-48a2-aef7-692dba67373f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:29.741546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20efeb2-5a05-48a2-aef7-692dba67373f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:29.764707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c20efeb2-5a05-48a2-aef7-692dba67373f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:29.766822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c20efeb2-5a05-48a2-aef7-692dba67373f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:29.804596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:29.864916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 463/926] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:29.887605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358dbfd4-4aa3-47f1-bc0e-f21ce776e89a None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:30.033176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358dbfd4-4aa3-47f1-bc0e-f21ce776e89a None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:30.074520 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a78e97-0559-4aca-b3bb-ac481299ef2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:30.075507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a78e97-0559-4aca-b3bb-ac481299ef2d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.075787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a78e97-0559-4aca-b3bb-ac481299ef2d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.076061 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a78e97-0559-4aca-b3bb-ac481299ef2d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.088912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a78e97-0559-4aca-b3bb-ac481299ef2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-62a78e97-0559-4aca-b3bb-ac481299ef2d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:30.098221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:30.099236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:30.100065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 464/927] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jul 19 20:41:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:30.105988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20efeb2-5a05-48a2-aef7-692dba67373f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:30.191875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3f03c-584e-4961-b18a-9974081072b5 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.192239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3f03c-584e-4961-b18a-9974081072b5 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.192703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3f03c-584e-4961-b18a-9974081072b5 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.200741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 465/928] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:30.207955 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf53cd6-fa7d-4ec0-9866-1e2389273563 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.208467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf53cd6-fa7d-4ec0-9866-1e2389273563 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.208909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf53cd6-fa7d-4ec0-9866-1e2389273563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.225478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20efeb2-5a05-48a2-aef7-692dba67373f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:30.239455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-2cf53cd6-fa7d-4ec0-9866-1e2389273563 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:30.240943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2cf53cd6-fa7d-4ec0-9866-1e2389273563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:30.277741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1e9ae-40cf-40cb-a1f4-262298279570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:30.278697 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1e9ae-40cf-40cb-a1f4-262298279570 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.279157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1e9ae-40cf-40cb-a1f4-262298279570 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.279589 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1e9ae-40cf-40cb-a1f4-262298279570 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.280596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1e9ae-40cf-40cb-a1f4-262298279570 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:30.281127 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1e9ae-40cf-40cb-a1f4-262298279570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:30.282901 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1e9ae-40cf-40cb-a1f4-262298279570 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:30.296030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 464/929] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jul 19 20:41:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:30.296652 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:30.301541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1e9ae-40cf-40cb-a1f4-262298279570 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:30.328113 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:30.336773 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 466/930] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:30.353344 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf53cd6-fa7d-4ec0-9866-1e2389273563 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:30.441896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1e9ae-40cf-40cb-a1f4-262298279570 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:30.489066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4e9c3-f1e3-45e1-b8cd-121360e19921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:30.489857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4e9c3-f1e3-45e1-b8cd-121360e19921 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.490152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4e9c3-f1e3-45e1-b8cd-121360e19921 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.490453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4e9c3-f1e3-45e1-b8cd-121360e19921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.494617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf53cd6-fa7d-4ec0-9866-1e2389273563 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:30.505729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4e9c3-f1e3-45e1-b8cd-121360e19921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:30.508035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4e9c3-f1e3-45e1-b8cd-121360e19921 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-a5b4e9c3-f1e3-45e1-b8cd-121360e19921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:30.533865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:30.534879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:30.535780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:30.535780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:30.535780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:30.535780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:30.536015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bd2e9-66cb-445a-bf06-1a4d5f1f1d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:30.536411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bd2e9-66cb-445a-bf06-1a4d5f1f1d1f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.536674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bd2e9-66cb-445a-bf06-1a4d5f1f1d1f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.536954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bd2e9-66cb-445a-bf06-1a4d5f1f1d1f None admin] PATH_INFO: `/v3/services/compute` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.537613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bd2e9-66cb-445a-bf06-1a4d5f1f1d1f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:30.538595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 465/931] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:30.544318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bd2e9-66cb-445a-bf06-1a4d5f1f1d1f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:30.545960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4e9c3-f1e3-45e1-b8cd-121360e19921 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-db6bd2e9-66cb-445a-bf06-1a4d5f1f1d1f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:30.558671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:30.559746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:41:30.560629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:30.564519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 467/932] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:30.566430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bd2e9-66cb-445a-bf06-1a4d5f1f1d1f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:30.615391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:30.649329 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:30.702267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4e9c3-f1e3-45e1-b8cd-121360e19921 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:30.710699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bd2e9-66cb-445a-bf06-1a4d5f1f1d1f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:30.753103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4335ce-95fe-4b5c-94d6-2471faebbdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:30.754945 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4335ce-95fe-4b5c-94d6-2471faebbdfb None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.755481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4335ce-95fe-4b5c-94d6-2471faebbdfb None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.756057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4335ce-95fe-4b5c-94d6-2471faebbdfb None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.756514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549df7c3-ca96-4970-bfc4-95ff87ad4d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:30.757181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549df7c3-ca96-4970-bfc4-95ff87ad4d94 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.757467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549df7c3-ca96-4970-bfc4-95ff87ad4d94 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.757654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4335ce-95fe-4b5c-94d6-2471faebbdfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:30.757762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549df7c3-ca96-4970-bfc4-95ff87ad4d94 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.758230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4335ce-95fe-4b5c-94d6-2471faebbdfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:30.758473 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549df7c3-ca96-4970-bfc4-95ff87ad4d94 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:30.758784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549df7c3-ca96-4970-bfc4-95ff87ad4d94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:30.760122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4335ce-95fe-4b5c-94d6-2471faebbdfb None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:30.760252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549df7c3-ca96-4970-bfc4-95ff87ad4d94 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:30.771468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 468/933] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:30.773094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81221642-bb31-46ef-a9e3-63006685022d None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.773391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81221642-bb31-46ef-a9e3-63006685022d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.773698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81221642-bb31-46ef-a9e3-63006685022d None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.775036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 469/934] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:30.776004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81221642-bb31-46ef-a9e3-63006685022d None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:30.779583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 466/935] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:30.781175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7827a-1ebf-4759-8e5b-cc20a03772f9 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.781468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7827a-1ebf-4759-8e5b-cc20a03772f9 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.781748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7827a-1ebf-4759-8e5b-cc20a03772f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.799556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-f4a7827a-1ebf-4759-8e5b-cc20a03772f9 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:30.800938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f4a7827a-1ebf-4759-8e5b-cc20a03772f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:30.884906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 467/936] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:30.889277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7827a-1ebf-4759-8e5b-cc20a03772f9 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:30.933823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81221642-bb31-46ef-a9e3-63006685022d None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:30.935612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:30.968638 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:30.983471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f1ad7-8f88-4f3e-a8c3-ee9c4b08ae93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:30.984153 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f1ad7-8f88-4f3e-a8c3-ee9c4b08ae93 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:30.984457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f1ad7-8f88-4f3e-a8c3-ee9c4b08ae93 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:30.984743 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f1ad7-8f88-4f3e-a8c3-ee9c4b08ae93 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:30.985519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f1ad7-8f88-4f3e-a8c3-ee9c4b08ae93 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:30.986416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f1ad7-8f88-4f3e-a8c3-ee9c4b08ae93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:30.987228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f1ad7-8f88-4f3e-a8c3-ee9c4b08ae93 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:31.004498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 470/937] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:31.004498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f1ad7-8f88-4f3e-a8c3-ee9c4b08ae93 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:31.028946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7827a-1ebf-4759-8e5b-cc20a03772f9 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:31.078707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e3872-f924-4ea2-9153-99a69a925efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:31.079615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e3872-f924-4ea2-9153-99a69a925efe None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:31.079901 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e3872-f924-4ea2-9153-99a69a925efe None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:31.080178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e3872-f924-4ea2-9153-99a69a925efe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:31.092100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e3872-f924-4ea2-9153-99a69a925efe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:31.093762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e3872-f924-4ea2-9153-99a69a925efe None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-f48e3872-f924-4ea2-9153-99a69a925efe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:31.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:31.128289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:31.129809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:31.129809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:31.129809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:31.129809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:31.129809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:31.129809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:31.129809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:31.129809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:31.129809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:31.129809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:31.129809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 468/938] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jul 19 20:41:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:31.136656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e3872-f924-4ea2-9153-99a69a925efe None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:31.173305 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f1ad7-8f88-4f3e-a8c3-ee9c4b08ae93 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:31.227368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba16d0-cabb-409a-8c7f-5064d7c640b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:31.228224 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba16d0-cabb-409a-8c7f-5064d7c640b6 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:31.228555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba16d0-cabb-409a-8c7f-5064d7c640b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:31.228837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba16d0-cabb-409a-8c7f-5064d7c640b6 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:31.229937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba16d0-cabb-409a-8c7f-5064d7c640b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:31.231442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba16d0-cabb-409a-8c7f-5064d7c640b6 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:31.232871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: WARNING keystone.server.flask.common [None req-5bba16d0-cabb-409a-8c7f-5064d7c640b6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 19 20:41:31.254570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.sql.core [None req-5bba16d0-cabb-409a-8c7f-5064d7c640b6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 20:41:31.254570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 19 20:41:31.254570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [parameters: {'id': 'f6f93f307cfa46c780b5c42526108f6a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 19 20:41:31.254570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86668) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-5bba16d0-cabb-409a-8c7f-5064d7c640b6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6f93f307cfa46c780b5c42526108f6a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6f93f307cfa46c780b5c42526108f6a. Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:41:31.262754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result.read() Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 19 20:41:31.263612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application yield resource Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise self.value Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application session.commit() Jul 19 20:41:31.264602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.session.flush() Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application compat.raise_( Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 20:41:31.265499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 20:41:31.266411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result.read() Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:41:31.267218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [parameters: {'id': 'f6f93f307cfa46c780b5c42526108f6a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:31.268078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:31.268943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:31.270050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:31.270050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 19 20:41:31.270050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 19 20:41:31.270050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:31.270050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:31.270050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 19 20:41:31.270050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 19 20:41:31.270050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6f93f307cfa46c780b5c42526108f6a. Jul 19 20:41:31.270050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:31.270050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 471/939] 10.222.0.85 () {60 vars in 1140 bytes} [Tue Jul 19 20:41:30 2022] POST /identity/v3/projects => generated 262 bytes in 270 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:31.275773 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba16d0-cabb-409a-8c7f-5064d7c640b6 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:31.306476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:31.320386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e3872-f924-4ea2-9153-99a69a925efe None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:31.344286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:31.377841 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae25a8-50b0-4f32-a593-f94d5cbc22a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:31.379667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae25a8-50b0-4f32-a593-f94d5cbc22a2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:31.380172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae25a8-50b0-4f32-a593-f94d5cbc22a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:31.381239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae25a8-50b0-4f32-a593-f94d5cbc22a2 None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:31.382864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae25a8-50b0-4f32-a593-f94d5cbc22a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:31.383763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae25a8-50b0-4f32-a593-f94d5cbc22a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:31.387211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae25a8-50b0-4f32-a593-f94d5cbc22a2 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:31.409404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba16d0-cabb-409a-8c7f-5064d7c640b6 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:31.432654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 469/940] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jul 19 20:41:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:31.439785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae25a8-50b0-4f32-a593-f94d5cbc22a2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:31.456873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea50e2-918a-4bb4-a3d9-186a96e0121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:31.457685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea50e2-918a-4bb4-a3d9-186a96e0121e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:31.458010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea50e2-918a-4bb4-a3d9-186a96e0121e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:31.458287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea50e2-918a-4bb4-a3d9-186a96e0121e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:31.466065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea50e2-918a-4bb4-a3d9-186a96e0121e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:31.467738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea50e2-918a-4bb4-a3d9-186a96e0121e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-f6ea50e2-918a-4bb4-a3d9-186a96e0121e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:31.484170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:31.485485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:31.486444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:31.486444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:31.486444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:31.486444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:31.486444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:31.486444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:31.486444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:31.486444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:31.486444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 472/941] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:31.492283 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea50e2-918a-4bb4-a3d9-186a96e0121e None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:31.613239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:31.620891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae25a8-50b0-4f32-a593-f94d5cbc22a2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:31.646601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea50e2-918a-4bb4-a3d9-186a96e0121e None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:31.669833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f73dce-109b-4034-92d3-54c73d64ce67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:31.671265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73dce-109b-4034-92d3-54c73d64ce67 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:31.671854 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73dce-109b-4034-92d3-54c73d64ce67 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:31.672321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f73dce-109b-4034-92d3-54c73d64ce67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:31.688874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f73dce-109b-4034-92d3-54c73d64ce67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:31.690574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f73dce-109b-4034-92d3-54c73d64ce67 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:31.706729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b839c9d-eea1-4683-97b5-c2f94564e46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:31.708235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b839c9d-eea1-4683-97b5-c2f94564e46b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:31.708974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b839c9d-eea1-4683-97b5-c2f94564e46b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:31.709663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b839c9d-eea1-4683-97b5-c2f94564e46b None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-f4f73dce-109b-4034-92d3-54c73d64ce67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:31.710912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:31.717158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:31.719240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:31.719240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:31.719240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:31.719240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:31.719240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 470/942] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:31.719713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b839c9d-eea1-4683-97b5-c2f94564e46b None admin] RBAC: Adding query filter params (name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:31.719713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b839c9d-eea1-4683-97b5-c2f94564e46b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:31.719713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b839c9d-eea1-4683-97b5-c2f94564e46b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:31.725118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f73dce-109b-4034-92d3-54c73d64ce67 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:31.748485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 473/943] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:31 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:31.775718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:31.883630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f73dce-109b-4034-92d3-54c73d64ce67 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:31.932911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fdfde-694e-4b24-ad84-59d786b24a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:31.933605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fdfde-694e-4b24-ad84-59d786b24a37 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:31.933892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fdfde-694e-4b24-ad84-59d786b24a37 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:31.934174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fdfde-694e-4b24-ad84-59d786b24a37 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:31.934918 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fdfde-694e-4b24-ad84-59d786b24a37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:31.935231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fdfde-694e-4b24-ad84-59d786b24a37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:31.936677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fdfde-694e-4b24-ad84-59d786b24a37 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:31.951925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 471/944] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:31.958115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b839c9d-eea1-4683-97b5-c2f94564e46b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:32.112259 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b839c9d-eea1-4683-97b5-c2f94564e46b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:32.140698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e729d-3b8b-4218-8155-41c9b8f5d89d None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:32.141035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e729d-3b8b-4218-8155-41c9b8f5d89d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:32.141386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e729d-3b8b-4218-8155-41c9b8f5d89d None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:32.143673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 472/945] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:32.150131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9c36b-0cf1-4dd4-8e0e-0c57d9b14e00 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:32.150423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9c36b-0cf1-4dd4-8e0e-0c57d9b14e00 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:32.150709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9c36b-0cf1-4dd4-8e0e-0c57d9b14e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:32.154795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d28c6-44e0-4fc6-9fd9-b65b46d7ea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:32.155592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d28c6-44e0-4fc6-9fd9-b65b46d7ea00 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:32.155877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d28c6-44e0-4fc6-9fd9-b65b46d7ea00 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:32.156156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d28c6-44e0-4fc6-9fd9-b65b46d7ea00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:32.163936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d28c6-44e0-4fc6-9fd9-b65b46d7ea00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:32.165442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d28c6-44e0-4fc6-9fd9-b65b46d7ea00 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:32.177021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-21c9c36b-0cf1-4dd4-8e0e-0c57d9b14e00 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:32.178448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-21c9c36b-0cf1-4dd4-8e0e-0c57d9b14e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-a19d28c6-44e0-4fc6-9fd9-b65b46d7ea00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:32.181000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:32.182136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:32.183136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:32.183136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:32.183136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:32.183136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:32.183136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:32.183136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:32.183136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:32.183136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:32.183136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 474/946] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:32.189907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d28c6-44e0-4fc6-9fd9-b65b46d7ea00 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:32.193316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:32.305660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 473/947] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:32.337202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9c36b-0cf1-4dd4-8e0e-0c57d9b14e00 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:32.353704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d28c6-44e0-4fc6-9fd9-b65b46d7ea00 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:32.381343 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:32.408266 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020ae7c-08fd-4c88-893b-f7c12e7c9a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:32.408988 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020ae7c-08fd-4c88-893b-f7c12e7c9a61 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:32.409325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020ae7c-08fd-4c88-893b-f7c12e7c9a61 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:32.409634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020ae7c-08fd-4c88-893b-f7c12e7c9a61 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:32.410963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020ae7c-08fd-4c88-893b-f7c12e7c9a61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:32.411273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020ae7c-08fd-4c88-893b-f7c12e7c9a61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:32.412772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020ae7c-08fd-4c88-893b-f7c12e7c9a61 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:32.430001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 475/948] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:32.436725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020ae7c-08fd-4c88-893b-f7c12e7c9a61 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:32.491848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9c36b-0cf1-4dd4-8e0e-0c57d9b14e00 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:32.529120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:32.548939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859c7a8-9397-433b-bf4d-05498c6e3b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:32.550648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859c7a8-9397-433b-bf4d-05498c6e3b41 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:32.550885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859c7a8-9397-433b-bf4d-05498c6e3b41 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:32.551183 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859c7a8-9397-433b-bf4d-05498c6e3b41 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:32.552151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859c7a8-9397-433b-bf4d-05498c6e3b41 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:32.553794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859c7a8-9397-433b-bf4d-05498c6e3b41 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-b859c7a8-9397-433b-bf4d-05498c6e3b41 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:32.570374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:32.571625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:32.572542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:32.577059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 474/949] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jul 19 20:41:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:32.578157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859c7a8-9397-433b-bf4d-05498c6e3b41 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:32.585039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020ae7c-08fd-4c88-893b-f7c12e7c9a61 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:32.629995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bd56c-82a4-4cc6-b613-afc66303cb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:32.631743 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bd56c-82a4-4cc6-b613-afc66303cb57 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:32.632623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bd56c-82a4-4cc6-b613-afc66303cb57 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:32.633200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bd56c-82a4-4cc6-b613-afc66303cb57 None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/e2f34a304e714f838d68dfdf214e4712/roles/c22095e125674044bf3d2fe3122eeacd` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:32.636440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bd56c-82a4-4cc6-b613-afc66303cb57 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=e2f34a304e714f838d68dfdf214e4712, role_id=c22095e125674044bf3d2fe3122eeacd)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:32.638060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bd56c-82a4-4cc6-b613-afc66303cb57 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:32.664420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 476/950] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:41:32 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/e2f34a304e714f838d68dfdf214e4712/roles/c22095e125674044bf3d2fe3122eeacd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:41:32.679513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:32.703678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859c7a8-9397-433b-bf4d-05498c6e3b41 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:32.746934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88ce85-133c-4dfe-ac6b-b07016a8979a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:32.747628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88ce85-133c-4dfe-ac6b-b07016a8979a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:32.747935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88ce85-133c-4dfe-ac6b-b07016a8979a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:32.748294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88ce85-133c-4dfe-ac6b-b07016a8979a None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:32.749218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88ce85-133c-4dfe-ac6b-b07016a8979a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:32.749532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88ce85-133c-4dfe-ac6b-b07016a8979a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:32.751048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88ce85-133c-4dfe-ac6b-b07016a8979a None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:32.759212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 475/951] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jul 19 20:41:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:32.765983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bd56c-82a4-4cc6-b613-afc66303cb57 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:32.892370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bd56c-82a4-4cc6-b613-afc66303cb57 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:32.919318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:32.939084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb121cef-6df8-4194-b6f7-afc25abf9752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:32.940074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb121cef-6df8-4194-b6f7-afc25abf9752 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:32.940592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb121cef-6df8-4194-b6f7-afc25abf9752 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:32.941569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb121cef-6df8-4194-b6f7-afc25abf9752 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:32.942574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb121cef-6df8-4194-b6f7-afc25abf9752 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:32.943066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb121cef-6df8-4194-b6f7-afc25abf9752 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:32.944621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb121cef-6df8-4194-b6f7-afc25abf9752 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:32.950708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 477/952] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jul 19 20:41:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:32.957389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88ce85-133c-4dfe-ac6b-b07016a8979a None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:33.080450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88ce85-133c-4dfe-ac6b-b07016a8979a None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:33.119746 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e9520-f489-4dd3-a19a-f3e7162a2458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:33.120821 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e9520-f489-4dd3-a19a-f3e7162a2458 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:33.121326 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e9520-f489-4dd3-a19a-f3e7162a2458 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:33.121793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e9520-f489-4dd3-a19a-f3e7162a2458 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:33.122785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e9520-f489-4dd3-a19a-f3e7162a2458 None admin] RBAC: Adding query filter params (interface=public, service_id=d52d6f652434478f8ac431075cdbfc49, region_id=RegionOne) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:33.123309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e9520-f489-4dd3-a19a-f3e7162a2458 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:33.125000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e9520-f489-4dd3-a19a-f3e7162a2458 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:33.132050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 476/953] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jul 19 20:41:32 2022] GET /identity/v3/endpoints?service_id=d52d6f652434478f8ac431075cdbfc49&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:33.141443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb121cef-6df8-4194-b6f7-afc25abf9752 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:33.257069 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb121cef-6df8-4194-b6f7-afc25abf9752 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:33.292152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf47176-f865-4f5f-97de-452875e00ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:33.292850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf47176-f865-4f5f-97de-452875e00ad1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:33.293139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf47176-f865-4f5f-97de-452875e00ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:33.293419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf47176-f865-4f5f-97de-452875e00ad1 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:33.294102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf47176-f865-4f5f-97de-452875e00ad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:33.295472 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf47176-f865-4f5f-97de-452875e00ad1 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:33.302727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 478/954] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jul 19 20:41:33 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:41:33.405144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3ac66-fb1e-4aed-8f4d-14cfb1b7a7f2 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:33.405645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3ac66-fb1e-4aed-8f4d-14cfb1b7a7f2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:33.405932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3ac66-fb1e-4aed-8f4d-14cfb1b7a7f2 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:33.407627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 477/955] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:33.413849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ee515-614f-4273-83e7-03633fa59edf None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:33.414578 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ee515-614f-4273-83e7-03633fa59edf None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:33.415302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ee515-614f-4273-83e7-03633fa59edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:33.435249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ba3ee515-614f-4273-83e7-03633fa59edf None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:33.437235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ba3ee515-614f-4273-83e7-03633fa59edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:33.496728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:33.523469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 479/956] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:33.548124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3ac66-fb1e-4aed-8f4d-14cfb1b7a7f2 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:33.563414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:33.674776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3ac66-fb1e-4aed-8f4d-14cfb1b7a7f2 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:33.719322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40f5b1-3423-4851-952c-c313cb0cea26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:33.720118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f5b1-3423-4851-952c-c313cb0cea26 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:33.720306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f5b1-3423-4851-952c-c313cb0cea26 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:33.720613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f5b1-3423-4851-952c-c313cb0cea26 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:33.721445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40f5b1-3423-4851-952c-c313cb0cea26 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:33.723427 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40f5b1-3423-4851-952c-c313cb0cea26 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:33.741196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 478/957] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:41:33 2022] POST /identity/v3/services => generated 242 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:34.118210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8e5dc-967a-498d-a5cc-351863a2b884 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:34.118881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8e5dc-967a-498d-a5cc-351863a2b884 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:34.118963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8e5dc-967a-498d-a5cc-351863a2b884 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:34.120969 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 480/958] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:34.127058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdde62-c2f9-419e-94f4-778561b3ae9e None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:34.127385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdde62-c2f9-419e-94f4-778561b3ae9e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:34.127624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdde62-c2f9-419e-94f4-778561b3ae9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:34.145851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-f0fdde62-c2f9-419e-94f4-778561b3ae9e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:34.147094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f0fdde62-c2f9-419e-94f4-778561b3ae9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:34.233274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:34.252680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 479/959] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:34.275878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8e5dc-967a-498d-a5cc-351863a2b884 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:34.409033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8e5dc-967a-498d-a5cc-351863a2b884 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:34.446651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a1447-93d8-4340-a010-5b7b878d74c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:34.447331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1447-93d8-4340-a010-5b7b878d74c5 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:34.447618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1447-93d8-4340-a010-5b7b878d74c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:34.447894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1447-93d8-4340-a010-5b7b878d74c5 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:34.448630 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a1447-93d8-4340-a010-5b7b878d74c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:34.450063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a1447-93d8-4340-a010-5b7b878d74c5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:34.464755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.sql.core [None req-107a1447-93d8-4340-a010-5b7b878d74c5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 20:41:34.464755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 19 20:41:34.464755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [parameters: {'user_id': 'feb026ad08cc4773b4f50a6c333279fb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 19 20:41:34.464755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86668) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-107a1447-93d8-4340-a010-5b7b878d74c5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result.read() Jul 19 20:41:34.481295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:41:34.482319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application yield resource Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application next(self.gen) Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise self.value Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application session.commit() Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 20:41:34.483287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.session.flush() Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application compat.raise_( Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 20:41:34.484208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 20:41:34.485366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application conn.query(q) Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application result.read() Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 20:41:34.486666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [parameters: {'user_id': 'feb026ad08cc4773b4f50a6c333279fb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:34.488050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 19 20:41:34.495173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 19 20:41:34.497585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 19 20:41:34.497585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 19 20:41:34.497585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 20:41:34.497585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 20:41:34.497585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 19 20:41:34.497585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:34.497585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 481/960] 10.222.0.85 () {60 vars in 1131 bytes} [Tue Jul 19 20:41:34 2022] POST /identity/v3/users => generated 160 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:34.503859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdde62-c2f9-419e-94f4-778561b3ae9e None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:34.646600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdde62-c2f9-419e-94f4-778561b3ae9e None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:34.694867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073c5f2-a38f-4f5e-be2f-49cf907bdb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:34.696250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073c5f2-a38f-4f5e-be2f-49cf907bdb87 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:34.696803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073c5f2-a38f-4f5e-be2f-49cf907bdb87 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:34.697506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073c5f2-a38f-4f5e-be2f-49cf907bdb87 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:34.708477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073c5f2-a38f-4f5e-be2f-49cf907bdb87 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:34.710329 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073c5f2-a38f-4f5e-be2f-49cf907bdb87 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-5073c5f2-a38f-4f5e-be2f-49cf907bdb87 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:34.726850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:34.728440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:34.729492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:34.729492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:34.729492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:34.729492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:34.729492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:34.729492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:34.729492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:34.729492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:34.729492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 20:41:34.729492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:34.735862 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 480/961] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:41:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:34.738959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a1447-93d8-4340-a010-5b7b878d74c5 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:34.796749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:34.866827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a1447-93d8-4340-a010-5b7b878d74c5 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:34.919206 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153385b4-5ca8-4cbe-a852-03f72fcc4fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:34.920587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153385b4-5ca8-4cbe-a852-03f72fcc4fe8 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:34.921136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153385b4-5ca8-4cbe-a852-03f72fcc4fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:34.921828 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153385b4-5ca8-4cbe-a852-03f72fcc4fe8 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:34.923022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153385b4-5ca8-4cbe-a852-03f72fcc4fe8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:34.923537 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153385b4-5ca8-4cbe-a852-03f72fcc4fe8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:34.925436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153385b4-5ca8-4cbe-a852-03f72fcc4fe8 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:34.971030 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 482/962] 10.222.0.85 () {56 vars in 1101 bytes} [Tue Jul 19 20:41:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:35.020554 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab618e42-10e3-435d-ba23-f8a6ba81c815 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:35.020897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab618e42-10e3-435d-ba23-f8a6ba81c815 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:35.021155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab618e42-10e3-435d-ba23-f8a6ba81c815 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:35.023020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 481/963] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:35.029918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5dc6-d607-443d-a8cd-2a9d441968ae None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:35.030200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5dc6-d607-443d-a8cd-2a9d441968ae None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:35.030523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5dc6-d607-443d-a8cd-2a9d441968ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:35.052042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0f4f5dc6-d607-443d-a8cd-2a9d441968ae None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:35.053910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0f4f5dc6-d607-443d-a8cd-2a9d441968ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:35.151921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 483/964] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:35.189578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab618e42-10e3-435d-ba23-f8a6ba81c815 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:35.280906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:35.361882 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab618e42-10e3-435d-ba23-f8a6ba81c815 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:35.422833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c279db-468e-4fef-8285-06494dc6e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:35.424199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c279db-468e-4fef-8285-06494dc6e7a3 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:35.424711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c279db-468e-4fef-8285-06494dc6e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:35.425642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c279db-468e-4fef-8285-06494dc6e7a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:35.447737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c279db-468e-4fef-8285-06494dc6e7a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-71c279db-468e-4fef-8285-06494dc6e7a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:35.464222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:35.465279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:35.466252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:35.468402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 482/965] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jul 19 20:41:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:35.475968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f5dc6-d607-443d-a8cd-2a9d441968ae None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:35.510558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:35.605836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f5dc6-d607-443d-a8cd-2a9d441968ae None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:35.648729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7783d93-adb9-42a6-b891-b9ab4258aa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:35.649423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7783d93-adb9-42a6-b891-b9ab4258aa9b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:35.649705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7783d93-adb9-42a6-b891-b9ab4258aa9b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:35.650055 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7783d93-adb9-42a6-b891-b9ab4258aa9b None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:35.651006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7783d93-adb9-42a6-b891-b9ab4258aa9b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:35.651313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7783d93-adb9-42a6-b891-b9ab4258aa9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:35.652784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7783d93-adb9-42a6-b891-b9ab4258aa9b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:35.667269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 484/966] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jul 19 20:41:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:35.668611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c279db-468e-4fef-8285-06494dc6e7a3 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:35.783982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70710df3-c6d9-4d4b-a97c-ab2501a014c6 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:35.784309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70710df3-c6d9-4d4b-a97c-ab2501a014c6 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:35.784616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70710df3-c6d9-4d4b-a97c-ab2501a014c6 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:35.786237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 485/967] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:35.791669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583343f-dd8e-45fc-af43-6bf7ba0fdaab None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:35.791967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583343f-dd8e-45fc-af43-6bf7ba0fdaab None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:35.792288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583343f-dd8e-45fc-af43-6bf7ba0fdaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:35.805374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c279db-468e-4fef-8285-06494dc6e7a3 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:35.810634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-6583343f-dd8e-45fc-af43-6bf7ba0fdaab None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:35.811879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6583343f-dd8e-45fc-af43-6bf7ba0fdaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:35.827565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:35.850203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42204983-43f9-442d-9eaa-cd996bf129f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:35.851011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42204983-43f9-442d-9eaa-cd996bf129f5 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:35.851343 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42204983-43f9-442d-9eaa-cd996bf129f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:35.851630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42204983-43f9-442d-9eaa-cd996bf129f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:35.864492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42204983-43f9-442d-9eaa-cd996bf129f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:35.865877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42204983-43f9-442d-9eaa-cd996bf129f5 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-42204983-43f9-442d-9eaa-cd996bf129f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:35.885794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:35.886837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:35.887752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:35.887752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:35.887752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:35.887752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:35.887752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 483/968] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:35.892944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42204983-43f9-442d-9eaa-cd996bf129f5 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:35.895088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 486/969] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:35.918348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583343f-dd8e-45fc-af43-6bf7ba0fdaab None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:35.944092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:36.014156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42204983-43f9-442d-9eaa-cd996bf129f5 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:36.055054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d456a2-3905-4d41-8334-9fa98b4744ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:36.056119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d456a2-3905-4d41-8334-9fa98b4744ed None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.056373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d456a2-3905-4d41-8334-9fa98b4744ed None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.056764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d456a2-3905-4d41-8334-9fa98b4744ed None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.057770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d456a2-3905-4d41-8334-9fa98b4744ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:36.058085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d456a2-3905-4d41-8334-9fa98b4744ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:36.059484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d456a2-3905-4d41-8334-9fa98b4744ed None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:36.060806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583343f-dd8e-45fc-af43-6bf7ba0fdaab None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:36.073843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 484/970] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:36.075105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd59a-3287-4f30-9304-da32307948a1 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.075392 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd59a-3287-4f30-9304-da32307948a1 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.075657 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd59a-3287-4f30-9304-da32307948a1 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.077188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 485/971] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:36.079201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd59a-3287-4f30-9304-da32307948a1 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:36.107883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206c733-c8f8-4037-a921-ae6cb9fec30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:36.108764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c733-c8f8-4037-a921-ae6cb9fec30c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.109054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c733-c8f8-4037-a921-ae6cb9fec30c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.109340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c733-c8f8-4037-a921-ae6cb9fec30c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.110048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206c733-c8f8-4037-a921-ae6cb9fec30c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:36.110909 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:36.111545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206c733-c8f8-4037-a921-ae6cb9fec30c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-8206c733-c8f8-4037-a921-ae6cb9fec30c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:36.127539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:36.129496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:36.130350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 487/972] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jul 19 20:41:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:36.131165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08e706-44d4-4d2d-8051-b1c5313dbb05 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.131233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08e706-44d4-4d2d-8051-b1c5313dbb05 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.131510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08e706-44d4-4d2d-8051-b1c5313dbb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.149527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-3e08e706-44d4-4d2d-8051-b1c5313dbb05 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:36.150763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3e08e706-44d4-4d2d-8051-b1c5313dbb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:36.226446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd59a-3287-4f30-9304-da32307948a1 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:36.227603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 488/973] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:36.228614 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08e706-44d4-4d2d-8051-b1c5313dbb05 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:36.261400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:36.274133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033d546-0871-42fe-b5a7-aa491e2e3df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:36.274968 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033d546-0871-42fe-b5a7-aa491e2e3df8 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.275240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033d546-0871-42fe-b5a7-aa491e2e3df8 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.275522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033d546-0871-42fe-b5a7-aa491e2e3df8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.283748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033d546-0871-42fe-b5a7-aa491e2e3df8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:36.285335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033d546-0871-42fe-b5a7-aa491e2e3df8 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-c033d546-0871-42fe-b5a7-aa491e2e3df8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:36.303238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:36.304595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 486/974] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jul 19 20:41:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:36.306230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033d546-0871-42fe-b5a7-aa491e2e3df8 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:36.379011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08e706-44d4-4d2d-8051-b1c5313dbb05 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:36.420146 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f795b95-0d37-451e-981b-4e6ef323ba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:36.421136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f795b95-0d37-451e-981b-4e6ef323ba68 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.421484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f795b95-0d37-451e-981b-4e6ef323ba68 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.421822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f795b95-0d37-451e-981b-4e6ef323ba68 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.422625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f795b95-0d37-451e-981b-4e6ef323ba68 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:36.422995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f795b95-0d37-451e-981b-4e6ef323ba68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:36.424956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f795b95-0d37-451e-981b-4e6ef323ba68 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:36.426183 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:36.433925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 489/975] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jul 19 20:41:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:36.437284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f795b95-0d37-451e-981b-4e6ef323ba68 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:36.442860 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033d546-0871-42fe-b5a7-aa491e2e3df8 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:36.487272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dd8a8-3614-4e14-92e5-a2052f9445ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:36.488826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dd8a8-3614-4e14-92e5-a2052f9445ed None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.489155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dd8a8-3614-4e14-92e5-a2052f9445ed None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.489422 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dd8a8-3614-4e14-92e5-a2052f9445ed None admin] PATH_INFO: `/v3/services/compute` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.490108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dd8a8-3614-4e14-92e5-a2052f9445ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:36.491576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dd8a8-3614-4e14-92e5-a2052f9445ed None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-ac1dd8a8-3614-4e14-92e5-a2052f9445ed None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:36.508056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:36.510137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:36.511760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 487/976] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:36.513348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dd8a8-3614-4e14-92e5-a2052f9445ed None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:36.566760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:36.591256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f795b95-0d37-451e-981b-4e6ef323ba68 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:36.643825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18140c-4c5e-4ed5-9761-271ea2e29eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:36.644471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18140c-4c5e-4ed5-9761-271ea2e29eda None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.644726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18140c-4c5e-4ed5-9761-271ea2e29eda None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.645003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18140c-4c5e-4ed5-9761-271ea2e29eda None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.645864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18140c-4c5e-4ed5-9761-271ea2e29eda None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:36.646177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18140c-4c5e-4ed5-9761-271ea2e29eda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:36.647658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18140c-4c5e-4ed5-9761-271ea2e29eda None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:36.651631 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dd8a8-3614-4e14-92e5-a2052f9445ed None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:36.674419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 490/977] 10.222.0.85 () {56 vars in 1149 bytes} [Tue Jul 19 20:41:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:36.674419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18140c-4c5e-4ed5-9761-271ea2e29eda None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:36.714372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f0c67-79bd-4f65-bbbe-31f73d8f09be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:36.715366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0c67-79bd-4f65-bbbe-31f73d8f09be None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.715649 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0c67-79bd-4f65-bbbe-31f73d8f09be None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.715939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0c67-79bd-4f65-bbbe-31f73d8f09be None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.716714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f0c67-79bd-4f65-bbbe-31f73d8f09be None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:36.717020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f0c67-79bd-4f65-bbbe-31f73d8f09be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:36.718457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f0c67-79bd-4f65-bbbe-31f73d8f09be None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:36.726923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 488/978] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jul 19 20:41:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:36.727824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f0c67-79bd-4f65-bbbe-31f73d8f09be None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:36.752434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:36.811352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18140c-4c5e-4ed5-9761-271ea2e29eda None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:36.859110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0c6bc-08e6-4f76-96b7-1aff87ca14bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:36.859809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0c6bc-08e6-4f76-96b7-1aff87ca14bc None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.860063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0c6bc-08e6-4f76-96b7-1aff87ca14bc None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.860356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0c6bc-08e6-4f76-96b7-1aff87ca14bc None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:36.861082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0c6bc-08e6-4f76-96b7-1aff87ca14bc None admin] RBAC: Adding query filter params (name=compute) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:36.861389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0c6bc-08e6-4f76-96b7-1aff87ca14bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:36.863330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0c6bc-08e6-4f76-96b7-1aff87ca14bc None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:36.864780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f0c67-79bd-4f65-bbbe-31f73d8f09be None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:36.870681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 491/979] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:36.871533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0c6bc-08e6-4f76-96b7-1aff87ca14bc None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:36.875297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:36.918729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578423f5-734b-48dd-8d48-6f4bf97ae183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:36.919396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578423f5-734b-48dd-8d48-6f4bf97ae183 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:36.919663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578423f5-734b-48dd-8d48-6f4bf97ae183 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:36.919927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578423f5-734b-48dd-8d48-6f4bf97ae183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:37.004664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0c6bc-08e6-4f76-96b7-1aff87ca14bc None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:37.006161 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578423f5-734b-48dd-8d48-6f4bf97ae183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:37.007652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578423f5-734b-48dd-8d48-6f4bf97ae183 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:37.033595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:37.045521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd96c71-c70b-4f7d-9ca7-1e75582a9287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:37.046232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd96c71-c70b-4f7d-9ca7-1e75582a9287 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:37.046578 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd96c71-c70b-4f7d-9ca7-1e75582a9287 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:37.047757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd96c71-c70b-4f7d-9ca7-1e75582a9287 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:37.047757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd96c71-c70b-4f7d-9ca7-1e75582a9287 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:37.049191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd96c71-c70b-4f7d-9ca7-1e75582a9287 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:37.076713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 492/980] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jul 19 20:41:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:37.077854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd96c71-c70b-4f7d-9ca7-1e75582a9287 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:37.172900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 489/981] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:36 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:41:37.174277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:37.179637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578423f5-734b-48dd-8d48-6f4bf97ae183 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:37.274203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd96c71-c70b-4f7d-9ca7-1e75582a9287 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:37.333868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe4a75-1c29-4d97-b28e-3b0269455341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:37.334637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4a75-1c29-4d97-b28e-3b0269455341 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:37.335113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4a75-1c29-4d97-b28e-3b0269455341 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:37.335247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe4a75-1c29-4d97-b28e-3b0269455341 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:37.337005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe4a75-1c29-4d97-b28e-3b0269455341 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:37.337005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe4a75-1c29-4d97-b28e-3b0269455341 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:37.338513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe4a75-1c29-4d97-b28e-3b0269455341 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:37.346350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 493/982] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:37.350310 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578423f5-734b-48dd-8d48-6f4bf97ae183 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:37.352307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe4a75-1c29-4d97-b28e-3b0269455341 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:37.376933 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:37.408092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4fb4c-cc9e-4199-ad3d-cf6f1d0f977f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:37.409149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4fb4c-cc9e-4199-ad3d-cf6f1d0f977f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:37.409496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4fb4c-cc9e-4199-ad3d-cf6f1d0f977f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:37.409800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4fb4c-cc9e-4199-ad3d-cf6f1d0f977f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:37.423161 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4fb4c-cc9e-4199-ad3d-cf6f1d0f977f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:37.424603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4fb4c-cc9e-4199-ad3d-cf6f1d0f977f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-dad4fb4c-cc9e-4199-ad3d-cf6f1d0f977f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:37.450438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:37.451434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:37.452627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:37.452627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:37.452627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:37.452627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:37.458412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 490/983] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:37.459256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4fb4c-cc9e-4199-ad3d-cf6f1d0f977f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:37.527118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe4a75-1c29-4d97-b28e-3b0269455341 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:37.551398 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:37.584420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddda1c6-e2b4-4d5f-a9a5-61e6b2adb2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:37.585501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddda1c6-e2b4-4d5f-a9a5-61e6b2adb2e2 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:37.585912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddda1c6-e2b4-4d5f-a9a5-61e6b2adb2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:37.586302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddda1c6-e2b4-4d5f-a9a5-61e6b2adb2e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:37.587228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddda1c6-e2b4-4d5f-a9a5-61e6b2adb2e2 None admin] RBAC: Adding query filter params (interface=public, service_id=095224575b0146d889ae867a44bb4b20, region_id=RegionOne) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:37.587549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddda1c6-e2b4-4d5f-a9a5-61e6b2adb2e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:37.588982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddda1c6-e2b4-4d5f-a9a5-61e6b2adb2e2 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:37.595230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 494/984] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jul 19 20:41:37 2022] GET /identity/v3/endpoints?service_id=095224575b0146d889ae867a44bb4b20&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:37.601296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddda1c6-e2b4-4d5f-a9a5-61e6b2adb2e2 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:37.604860 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4fb4c-cc9e-4199-ad3d-cf6f1d0f977f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:37.658276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728c398-aec2-43b9-8ed1-5605df05b62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:37.659045 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728c398-aec2-43b9-8ed1-5605df05b62c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:37.659333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728c398-aec2-43b9-8ed1-5605df05b62c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:37.659734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728c398-aec2-43b9-8ed1-5605df05b62c None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:37.660501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728c398-aec2-43b9-8ed1-5605df05b62c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:37.660814 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728c398-aec2-43b9-8ed1-5605df05b62c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:37.662399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728c398-aec2-43b9-8ed1-5605df05b62c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:37.689574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 491/985] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:37.692448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728c398-aec2-43b9-8ed1-5605df05b62c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:37.744448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:37.765119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddda1c6-e2b4-4d5f-a9a5-61e6b2adb2e2 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:37.811321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb493cd9-b4c6-4690-869a-43bf2d83005d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:37.812013 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb493cd9-b4c6-4690-869a-43bf2d83005d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:37.812300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb493cd9-b4c6-4690-869a-43bf2d83005d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:37.812613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb493cd9-b4c6-4690-869a-43bf2d83005d None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:37.813308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb493cd9-b4c6-4690-869a-43bf2d83005d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:37.814850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb493cd9-b4c6-4690-869a-43bf2d83005d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:37.832366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 495/986] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jul 19 20:41:37 2022] GET /identity/v3/services => generated 1727 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:41:37.860002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728c398-aec2-43b9-8ed1-5605df05b62c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:37.918694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757063a-bcea-4005-9ba0-f3442bd8d225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:37.919580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757063a-bcea-4005-9ba0-f3442bd8d225 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:37.919982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757063a-bcea-4005-9ba0-f3442bd8d225 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:37.920415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757063a-bcea-4005-9ba0-f3442bd8d225 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:37.929519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757063a-bcea-4005-9ba0-f3442bd8d225 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:37.930989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757063a-bcea-4005-9ba0-f3442bd8d225 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-7757063a-bcea-4005-9ba0-f3442bd8d225 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:37.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:37.948877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:37.953437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:37.953437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:37.953437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:37.953437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:37.953437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:37.953437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:37.953437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:37.953437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:37.953437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 492/987] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:37.957448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb493cd9-b4c6-4690-869a-43bf2d83005d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:38.013141 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:38.070898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb493cd9-b4c6-4690-869a-43bf2d83005d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:38.105196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edccb9-8da0-4606-af4e-eacbed7b4a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:38.105865 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edccb9-8da0-4606-af4e-eacbed7b4a5b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:38.106154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edccb9-8da0-4606-af4e-eacbed7b4a5b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:38.106435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edccb9-8da0-4606-af4e-eacbed7b4a5b None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:38.107224 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edccb9-8da0-4606-af4e-eacbed7b4a5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:38.107540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edccb9-8da0-4606-af4e-eacbed7b4a5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:38.109013 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edccb9-8da0-4606-af4e-eacbed7b4a5b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:38.122477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 496/988] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:38.128598 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757063a-bcea-4005-9ba0-f3442bd8d225 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:38.241016 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:38.256933 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757063a-bcea-4005-9ba0-f3442bd8d225 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:38.301962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d84d3-e6b4-4d73-812a-18155b33f09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:38.302658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d84d3-e6b4-4d73-812a-18155b33f09f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:38.302947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d84d3-e6b4-4d73-812a-18155b33f09f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:38.303227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d84d3-e6b4-4d73-812a-18155b33f09f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:38.304089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d84d3-e6b4-4d73-812a-18155b33f09f None admin] RBAC: Adding query filter params (role.id=c22095e125674044bf3d2fe3122eeacd, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=e2f34a304e714f838d68dfdf214e4712) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:38.304424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d84d3-e6b4-4d73-812a-18155b33f09f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:38.306387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d84d3-e6b4-4d73-812a-18155b33f09f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:38.313847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 493/989] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:38 2022] GET /identity/v3/role_assignments?user.id=e2f34a304e714f838d68dfdf214e4712&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=c22095e125674044bf3d2fe3122eeacd => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:39.266277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c670ce6-58d9-43dd-924d-50ee8d491957 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:39.266817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c670ce6-58d9-43dd-924d-50ee8d491957 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:39.266817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c670ce6-58d9-43dd-924d-50ee8d491957 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:39.268374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 497/990] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:39.274000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571a321-a04d-45f0-ba18-0af866f410bc None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:39.274454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571a321-a04d-45f0-ba18-0af866f410bc None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:39.274568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571a321-a04d-45f0-ba18-0af866f410bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:39.292839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-5571a321-a04d-45f0-ba18-0af866f410bc None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:39.294265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-5571a321-a04d-45f0-ba18-0af866f410bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:39.364880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 494/991] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:39.387192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c670ce6-58d9-43dd-924d-50ee8d491957 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:39.496818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c670ce6-58d9-43dd-924d-50ee8d491957 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:39.506539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:39.537306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b12642-8026-478e-8bc3-f1cf9bb955fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:39.538127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b12642-8026-478e-8bc3-f1cf9bb955fb None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:39.538410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b12642-8026-478e-8bc3-f1cf9bb955fb None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:39.538685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b12642-8026-478e-8bc3-f1cf9bb955fb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:39.539364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b12642-8026-478e-8bc3-f1cf9bb955fb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:39.540980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b12642-8026-478e-8bc3-f1cf9bb955fb None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-19b12642-8026-478e-8bc3-f1cf9bb955fb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:39.553959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:39.555005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:39.555847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:39.557618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 498/992] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jul 19 20:41:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:39.561550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571a321-a04d-45f0-ba18-0af866f410bc None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:39.647945 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:39.669606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571a321-a04d-45f0-ba18-0af866f410bc None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:39.705784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2990d96-f977-44a1-9efa-a786c6123337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:39.707057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2990d96-f977-44a1-9efa-a786c6123337 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:39.707348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2990d96-f977-44a1-9efa-a786c6123337 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:39.707629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2990d96-f977-44a1-9efa-a786c6123337 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:39.708436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2990d96-f977-44a1-9efa-a786c6123337 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:39.708771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2990d96-f977-44a1-9efa-a786c6123337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:39.710150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2990d96-f977-44a1-9efa-a786c6123337 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:39.715252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 495/993] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jul 19 20:41:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:39.721227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b12642-8026-478e-8bc3-f1cf9bb955fb None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:39.826879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b12642-8026-478e-8bc3-f1cf9bb955fb None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:39.862653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265e680-e4e0-4439-91cc-787e2c8827b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:39.863322 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265e680-e4e0-4439-91cc-787e2c8827b5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:39.863609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265e680-e4e0-4439-91cc-787e2c8827b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:39.863887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265e680-e4e0-4439-91cc-787e2c8827b5 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:39.864631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265e680-e4e0-4439-91cc-787e2c8827b5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:39.864947 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265e680-e4e0-4439-91cc-787e2c8827b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:39.866316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265e680-e4e0-4439-91cc-787e2c8827b5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:39.871759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 499/994] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jul 19 20:41:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:39.877562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2990d96-f977-44a1-9efa-a786c6123337 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:39.929003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72717553-e50b-4784-a3b8-0d2a8e3a8803 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:39.929298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72717553-e50b-4784-a3b8-0d2a8e3a8803 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:39.929579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72717553-e50b-4784-a3b8-0d2a8e3a8803 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:39.930998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 500/995] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:39.937091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f37dd-dbfb-4c99-9436-129dd5ef46cd None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:39.937465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f37dd-dbfb-4c99-9436-129dd5ef46cd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:39.937790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f37dd-dbfb-4c99-9436-129dd5ef46cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:39.957433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-354f37dd-dbfb-4c99-9436-129dd5ef46cd None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:39.958933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-354f37dd-dbfb-4c99-9436-129dd5ef46cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:39.988179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:39.997008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2990d96-f977-44a1-9efa-a786c6123337 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:40.036045 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4b311-37cf-485d-8da0-9f872920484e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:40.036962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4b311-37cf-485d-8da0-9f872920484e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.037080 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 501/996] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:40.040624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4b311-37cf-485d-8da0-9f872920484e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.040929 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4b311-37cf-485d-8da0-9f872920484e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.041748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4b311-37cf-485d-8da0-9f872920484e None admin] RBAC: Adding query filter params (interface=admin, service_id=d52d6f652434478f8ac431075cdbfc49, region_id=RegionOne) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:40.042064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4b311-37cf-485d-8da0-9f872920484e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:40.043451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4b311-37cf-485d-8da0-9f872920484e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:40.049218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 496/997] 10.222.0.85 () {56 vars in 1249 bytes} [Tue Jul 19 20:41:39 2022] GET /identity/v3/endpoints?service_id=d52d6f652434478f8ac431075cdbfc49&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:40.055073 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f37dd-dbfb-4c99-9436-129dd5ef46cd None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:40.063654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4b311-37cf-485d-8da0-9f872920484e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:40.077652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:40.165123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f37dd-dbfb-4c99-9436-129dd5ef46cd None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:40.178715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4b311-37cf-485d-8da0-9f872920484e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:40.207051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c95b5-df8d-4485-a7fe-3d45191b4bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:40.208090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c95b5-df8d-4485-a7fe-3d45191b4bbf None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.208578 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c95b5-df8d-4485-a7fe-3d45191b4bbf None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.208860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c95b5-df8d-4485-a7fe-3d45191b4bbf None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.209555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c95b5-df8d-4485-a7fe-3d45191b4bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:40.210997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c95b5-df8d-4485-a7fe-3d45191b4bbf None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:40.217597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e614f7-bcee-4aa5-8999-10183f567bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:40.228844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 502/998] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jul 19 20:41:40 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:41:40.229056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e614f7-bcee-4aa5-8999-10183f567bb3 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.229056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e614f7-bcee-4aa5-8999-10183f567bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.229056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e614f7-bcee-4aa5-8999-10183f567bb3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.229056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e614f7-bcee-4aa5-8999-10183f567bb3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:40.229056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e614f7-bcee-4aa5-8999-10183f567bb3 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-45e614f7-bcee-4aa5-8999-10183f567bb3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:40.239881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:40.241145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 20:41:40.242002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:40.244713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 497/999] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:40.247389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c95b5-df8d-4485-a7fe-3d45191b4bbf None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:40.281534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:40.364823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c95b5-df8d-4485-a7fe-3d45191b4bbf None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:40.401606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02be77d-849d-43ac-ada4-099ec8cdbf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:40.402284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02be77d-849d-43ac-ada4-099ec8cdbf3c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.402574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02be77d-849d-43ac-ada4-099ec8cdbf3c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.402858 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02be77d-849d-43ac-ada4-099ec8cdbf3c None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.403568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02be77d-849d-43ac-ada4-099ec8cdbf3c None admin] RBAC: Adding query filter params (name=compute) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:40.403878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02be77d-849d-43ac-ada4-099ec8cdbf3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:40.405275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02be77d-849d-43ac-ada4-099ec8cdbf3c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:40.410874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 503/1000] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:40.417025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e614f7-bcee-4aa5-8999-10183f567bb3 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:40.437666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de3aa7-a951-42dd-872a-4ce727349a4f None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.437991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de3aa7-a951-42dd-872a-4ce727349a4f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.438272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de3aa7-a951-42dd-872a-4ce727349a4f None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.440177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 504/1001] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:40.445500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e793b69-bd4b-46b2-a0ef-79e3b2ebb813 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.445796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e793b69-bd4b-46b2-a0ef-79e3b2ebb813 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.446066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e793b69-bd4b-46b2-a0ef-79e3b2ebb813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.463940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-8e793b69-bd4b-46b2-a0ef-79e3b2ebb813 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:40.465186 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8e793b69-bd4b-46b2-a0ef-79e3b2ebb813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:40.497990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:40.525324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e614f7-bcee-4aa5-8999-10183f567bb3 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:40.540233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 505/1002] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:40.560439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8024-a9c0-4785-add9-99e3c745ece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:40.561123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8024-a9c0-4785-add9-99e3c745ece9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.561406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8024-a9c0-4785-add9-99e3c745ece9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.561685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8024-a9c0-4785-add9-99e3c745ece9 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.562484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e8024-a9c0-4785-add9-99e3c745ece9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:40.562715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e8024-a9c0-4785-add9-99e3c745ece9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:40.564007 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e793b69-bd4b-46b2-a0ef-79e3b2ebb813 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:40.564151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e8024-a9c0-4785-add9-99e3c745ece9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:40.570167 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 498/1003] 10.222.0.85 () {56 vars in 1112 bytes} [Tue Jul 19 20:41:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:40.577492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8024-a9c0-4785-add9-99e3c745ece9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:40.596660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:40.687529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e793b69-bd4b-46b2-a0ef-79e3b2ebb813 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:40.690101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8024-a9c0-4785-add9-99e3c745ece9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:40.724297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595d221-e292-4868-b9dd-cdcf30a04839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:40.724806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595d221-e292-4868-b9dd-cdcf30a04839 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.725094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595d221-e292-4868-b9dd-cdcf30a04839 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.725370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595d221-e292-4868-b9dd-cdcf30a04839 None admin] PATH_INFO: `/v3/services/image` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.726024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595d221-e292-4868-b9dd-cdcf30a04839 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:40.727427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595d221-e292-4868-b9dd-cdcf30a04839 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:40.727530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ef7ec-bdb9-4da5-88b1-5ea2ad835d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:40.728530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ef7ec-bdb9-4da5-88b1-5ea2ad835d8f None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.728816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ef7ec-bdb9-4da5-88b1-5ea2ad835d8f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.729086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ef7ec-bdb9-4da5-88b1-5ea2ad835d8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.729827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ef7ec-bdb9-4da5-88b1-5ea2ad835d8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:40.731187 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ef7ec-bdb9-4da5-88b1-5ea2ad835d8f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:40.738837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-8595d221-e292-4868-b9dd-cdcf30a04839 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:40.739895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:40.740940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:40.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 506/1004] 10.222.0.85 () {56 vars in 1105 bytes} [Tue Jul 19 20:41:40 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:40.747457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595d221-e292-4868-b9dd-cdcf30a04839 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:40.756198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 499/1005] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jul 19 20:41:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:41:40.851689 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:40.869196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595d221-e292-4868-b9dd-cdcf30a04839 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:40.915292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20102d-2f19-4e53-b5bf-acff9c12c914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:40.915959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20102d-2f19-4e53-b5bf-acff9c12c914 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:40.916247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20102d-2f19-4e53-b5bf-acff9c12c914 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:40.916631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20102d-2f19-4e53-b5bf-acff9c12c914 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:40.917338 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20102d-2f19-4e53-b5bf-acff9c12c914 None admin] RBAC: Adding query filter params (name=image) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:40.917655 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20102d-2f19-4e53-b5bf-acff9c12c914 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:40.919118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20102d-2f19-4e53-b5bf-acff9c12c914 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:40.924444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 507/1006] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:40.930266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ef7ec-bdb9-4da5-88b1-5ea2ad835d8f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:41.043651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ef7ec-bdb9-4da5-88b1-5ea2ad835d8f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:41.080073 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a45e0-b6a2-4c4c-b2cc-7baa099943af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:41.080775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a45e0-b6a2-4c4c-b2cc-7baa099943af None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:41.081057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a45e0-b6a2-4c4c-b2cc-7baa099943af None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:41.081330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a45e0-b6a2-4c4c-b2cc-7baa099943af None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:41.082644 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a45e0-b6a2-4c4c-b2cc-7baa099943af None admin] RBAC: Adding query filter params (type=image) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:41.082952 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a45e0-b6a2-4c4c-b2cc-7baa099943af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:41.084338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a45e0-b6a2-4c4c-b2cc-7baa099943af None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:41.089537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 500/1007] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:40 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:42.289049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996df0-1643-4046-9222-636bbcea3c0a None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:42.289588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996df0-1643-4046-9222-636bbcea3c0a None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:42.289588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996df0-1643-4046-9222-636bbcea3c0a None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:42.291426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 508/1008] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:42.296624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873a3fe-d803-4b75-9e41-0a1abb5d5a9f None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:42.297018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873a3fe-d803-4b75-9e41-0a1abb5d5a9f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:42.297199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873a3fe-d803-4b75-9e41-0a1abb5d5a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:42.315082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d873a3fe-d803-4b75-9e41-0a1abb5d5a9f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:42.316912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d873a3fe-d803-4b75-9e41-0a1abb5d5a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:42.371196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:42.392417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 501/1009] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:42 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:42.411574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44996df0-1643-4046-9222-636bbcea3c0a None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:42.517879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44996df0-1643-4046-9222-636bbcea3c0a None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:42.553864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdb381-91e9-40a2-94da-2f90cc50d3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:42.554681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdb381-91e9-40a2-94da-2f90cc50d3ef None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:42.554967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdb381-91e9-40a2-94da-2f90cc50d3ef None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:42.555244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdb381-91e9-40a2-94da-2f90cc50d3ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:42.555896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdb381-91e9-40a2-94da-2f90cc50d3ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:42.557538 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdb381-91e9-40a2-94da-2f90cc50d3ef None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-08bdb381-91e9-40a2-94da-2f90cc50d3ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:42.570494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:42.572127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 20:41:42.573047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:42.576413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 509/1010] 10.222.0.85 () {56 vars in 1126 bytes} [Tue Jul 19 20:41:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:42.578222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873a3fe-d803-4b75-9e41-0a1abb5d5a9f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:42.682841 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873a3fe-d803-4b75-9e41-0a1abb5d5a9f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:42.718242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021899f5-30f6-4133-aa4c-058fdb5877ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:42.719212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021899f5-30f6-4133-aa4c-058fdb5877ae None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:42.719713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021899f5-30f6-4133-aa4c-058fdb5877ae None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:42.720222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021899f5-30f6-4133-aa4c-058fdb5877ae None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:42.721174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021899f5-30f6-4133-aa4c-058fdb5877ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:42.721658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021899f5-30f6-4133-aa4c-058fdb5877ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:42.723224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021899f5-30f6-4133-aa4c-058fdb5877ae None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:42.733422 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 502/1011] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jul 19 20:41:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:42.735081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdb381-91e9-40a2-94da-2f90cc50d3ef None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:42.752876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:42.874026 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdb381-91e9-40a2-94da-2f90cc50d3ef None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:42.913814 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61957d6e-d73c-4720-a1e3-31705073fcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:42.914824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61957d6e-d73c-4720-a1e3-31705073fcc2 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:42.915421 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61957d6e-d73c-4720-a1e3-31705073fcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:42.915901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61957d6e-d73c-4720-a1e3-31705073fcc2 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:42.917092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61957d6e-d73c-4720-a1e3-31705073fcc2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:42.917601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61957d6e-d73c-4720-a1e3-31705073fcc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:42.919187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61957d6e-d73c-4720-a1e3-31705073fcc2 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:42.928456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba9b09-d4a4-48b7-93d0-8b17062ecbf2 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:42.928456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba9b09-d4a4-48b7-93d0-8b17062ecbf2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:42.928456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba9b09-d4a4-48b7-93d0-8b17062ecbf2 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:42.928456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 503/1012] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:42.928790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 510/1013] 10.222.0.85 () {56 vars in 1122 bytes} [Tue Jul 19 20:41:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:42.931450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba9b09-d4a4-48b7-93d0-8b17062ecbf2 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:42.932794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379c8aa-2488-47a1-9a52-293546252c55 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:42.933086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379c8aa-2488-47a1-9a52-293546252c55 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:42.933360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379c8aa-2488-47a1-9a52-293546252c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:42.953023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-a379c8aa-2488-47a1-9a52-293546252c55 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:42.954519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a379c8aa-2488-47a1-9a52-293546252c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:42.991612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:43.031063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 511/1014] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:42 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:43.044417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba9b09-d4a4-48b7-93d0-8b17062ecbf2 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:43.055174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379c8aa-2488-47a1-9a52-293546252c55 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:43.058743 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:43.081027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9f1ad-604b-4d07-80ca-3068625a0615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:43.081722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9f1ad-604b-4d07-80ca-3068625a0615 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.082008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9f1ad-604b-4d07-80ca-3068625a0615 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.082270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9f1ad-604b-4d07-80ca-3068625a0615 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.083025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9f1ad-604b-4d07-80ca-3068625a0615 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:43.084453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9f1ad-604b-4d07-80ca-3068625a0615 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:43.099857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 504/1015] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jul 19 20:41:42 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:41:43.121045 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644e60b-1e98-4ab7-9342-abd4b6600856 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.121442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644e60b-1e98-4ab7-9342-abd4b6600856 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.121787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644e60b-1e98-4ab7-9342-abd4b6600856 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.123572 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 505/1016] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:43.130532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2755a10-b69c-4e36-aeac-b8f7f3f85db3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.130838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2755a10-b69c-4e36-aeac-b8f7f3f85db3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.131036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2755a10-b69c-4e36-aeac-b8f7f3f85db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.148139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-a2755a10-b69c-4e36-aeac-b8f7f3f85db3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:43.149397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a2755a10-b69c-4e36-aeac-b8f7f3f85db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:43.184014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379c8aa-2488-47a1-9a52-293546252c55 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:43.238390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a6fa4-5404-4280-b393-22f21f73fc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:43.239083 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 506/1017] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:43 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:43.239214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a6fa4-5404-4280-b393-22f21f73fc49 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.239490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a6fa4-5404-4280-b393-22f21f73fc49 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.239768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a6fa4-5404-4280-b393-22f21f73fc49 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.252047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a6fa4-5404-4280-b393-22f21f73fc49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-853a6fa4-5404-4280-b393-22f21f73fc49 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:43.261120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:43.262275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:43.263873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 512/1018] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jul 19 20:41:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:43.264814 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2755a10-b69c-4e36-aeac-b8f7f3f85db3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:43.268533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a6fa4-5404-4280-b393-22f21f73fc49 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:43.316714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:43.340750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:43.383206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2755a10-b69c-4e36-aeac-b8f7f3f85db3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:43.387527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a6fa4-5404-4280-b393-22f21f73fc49 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:43.421070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac612a1-df06-48a0-8484-70a79c4c44ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:43.422155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac612a1-df06-48a0-8484-70a79c4c44ee None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.422436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac612a1-df06-48a0-8484-70a79c4c44ee None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.422717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac612a1-df06-48a0-8484-70a79c4c44ee None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.423441 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac612a1-df06-48a0-8484-70a79c4c44ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:43.424867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac612a1-df06-48a0-8484-70a79c4c44ee None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:43.426043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f112e7-8c9a-46cd-b194-d95036bd4546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:43.426715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f112e7-8c9a-46cd-b194-d95036bd4546 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.427001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f112e7-8c9a-46cd-b194-d95036bd4546 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.427281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f112e7-8c9a-46cd-b194-d95036bd4546 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.428034 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f112e7-8c9a-46cd-b194-d95036bd4546 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:43.428364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f112e7-8c9a-46cd-b194-d95036bd4546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:43.429792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f112e7-8c9a-46cd-b194-d95036bd4546 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:43.439593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 513/1019] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jul 19 20:41:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:43.440760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 507/1020] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:41:43 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:43.449568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f112e7-8c9a-46cd-b194-d95036bd4546 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:43.569450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f112e7-8c9a-46cd-b194-d95036bd4546 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:43.594640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:43.616127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ebeb2-d147-4f88-9920-350974062122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:43.616985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ebeb2-d147-4f88-9920-350974062122 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.617269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ebeb2-d147-4f88-9920-350974062122 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.617546 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ebeb2-d147-4f88-9920-350974062122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.629557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ebeb2-d147-4f88-9920-350974062122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:43.631193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ebeb2-d147-4f88-9920-350974062122 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-1b1ebeb2-d147-4f88-9920-350974062122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:43.649900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:43.650956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:43.651876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:43.651876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:43.651876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:43.651876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:43.651876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 514/1021] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:43.656784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac612a1-df06-48a0-8484-70a79c4c44ee None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:43.759129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:43.765303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac612a1-df06-48a0-8484-70a79c4c44ee None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:43.808432 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dc1cf-b07c-4eab-a0b2-8312a62457fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:43.809415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dc1cf-b07c-4eab-a0b2-8312a62457fc None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.809881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dc1cf-b07c-4eab-a0b2-8312a62457fc None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.810367 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dc1cf-b07c-4eab-a0b2-8312a62457fc None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.811271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908dc1cf-b07c-4eab-a0b2-8312a62457fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:43.811760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908dc1cf-b07c-4eab-a0b2-8312a62457fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:43.813335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908dc1cf-b07c-4eab-a0b2-8312a62457fc None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:43.831725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 508/1022] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:43.833302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ebeb2-d147-4f88-9920-350974062122 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:43.947256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ebeb2-d147-4f88-9920-350974062122 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:43.983975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0b01e-7f32-4b52-b562-d5f878e1cedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:43.984909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0b01e-7f32-4b52-b562-d5f878e1cedd None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:43.985226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0b01e-7f32-4b52-b562-d5f878e1cedd None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:43.985532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0b01e-7f32-4b52-b562-d5f878e1cedd None admin] PATH_INFO: `/v3/users/nova` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:43.993627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0b01e-7f32-4b52-b562-d5f878e1cedd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:43.995229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0b01e-7f32-4b52-b562-d5f878e1cedd None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-c7f0b01e-7f32-4b52-b562-d5f878e1cedd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:44.011957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:44.013036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:44.018666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:44.018666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:44.018666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:44.018666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:44.018666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:44.018666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:44.018666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:44.018666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:44.018666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:41:44.018666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:44.018666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 515/1023] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jul 19 20:41:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:44.019363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dc1cf-b07c-4eab-a0b2-8312a62457fc None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:44.129390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dc1cf-b07c-4eab-a0b2-8312a62457fc None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:44.165440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686388ae-d8e0-4a7b-9bf8-db54dc92c902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:44.166169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686388ae-d8e0-4a7b-9bf8-db54dc92c902 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:44.166490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686388ae-d8e0-4a7b-9bf8-db54dc92c902 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:44.166769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686388ae-d8e0-4a7b-9bf8-db54dc92c902 None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:44.167611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686388ae-d8e0-4a7b-9bf8-db54dc92c902 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:44.167925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686388ae-d8e0-4a7b-9bf8-db54dc92c902 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:44.169393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686388ae-d8e0-4a7b-9bf8-db54dc92c902 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:44.192867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 509/1024] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jul 19 20:41:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:44.194392 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0b01e-7f32-4b52-b562-d5f878e1cedd None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:44.242628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:44.324579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0b01e-7f32-4b52-b562-d5f878e1cedd None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:44.359597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580ffbe-a1b6-4291-a5d1-f01170490c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:44.360402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580ffbe-a1b6-4291-a5d1-f01170490c8a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:44.360651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580ffbe-a1b6-4291-a5d1-f01170490c8a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:44.360970 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580ffbe-a1b6-4291-a5d1-f01170490c8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:44.429919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580ffbe-a1b6-4291-a5d1-f01170490c8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:44.431367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580ffbe-a1b6-4291-a5d1-f01170490c8a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:44.460810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:44.537421 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 516/1025] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:44 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:41:44.539011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686388ae-d8e0-4a7b-9bf8-db54dc92c902 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:44.559259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:44.647904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686388ae-d8e0-4a7b-9bf8-db54dc92c902 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:44.682374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1ef6a-ae60-48e1-bcbd-a397dc34cf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:44.683177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1ef6a-ae60-48e1-bcbd-a397dc34cf54 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:44.683465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1ef6a-ae60-48e1-bcbd-a397dc34cf54 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:44.683745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1ef6a-ae60-48e1-bcbd-a397dc34cf54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:44.694439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1ef6a-ae60-48e1-bcbd-a397dc34cf54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:44.695896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1ef6a-ae60-48e1-bcbd-a397dc34cf54 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-9ca1ef6a-ae60-48e1-bcbd-a397dc34cf54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:44.713433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:44.714374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:44.715275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:44.715275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:44.715275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:44.715275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:44.715275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 510/1026] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:44.721299 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580ffbe-a1b6-4291-a5d1-f01170490c8a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:44.825576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580ffbe-a1b6-4291-a5d1-f01170490c8a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:44.855715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:44.864449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6efb6a-2645-48e2-9b06-978d2e92c890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:44.865296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6efb6a-2645-48e2-9b06-978d2e92c890 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:44.865618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6efb6a-2645-48e2-9b06-978d2e92c890 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:44.865934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6efb6a-2645-48e2-9b06-978d2e92c890 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:44.866665 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6efb6a-2645-48e2-9b06-978d2e92c890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:44.866976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6efb6a-2645-48e2-9b06-978d2e92c890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:44.868358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6efb6a-2645-48e2-9b06-978d2e92c890 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:44.884683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 517/1027] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:44.886139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1ef6a-ae60-48e1-bcbd-a397dc34cf54 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:44.954180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:44.990146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1ef6a-ae60-48e1-bcbd-a397dc34cf54 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:45.025793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1eda70-be5e-42c4-a560-e5a12cee0895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:45.026597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1eda70-be5e-42c4-a560-e5a12cee0895 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.026890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1eda70-be5e-42c4-a560-e5a12cee0895 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.027167 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1eda70-be5e-42c4-a560-e5a12cee0895 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.034088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1eda70-be5e-42c4-a560-e5a12cee0895 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:45.036064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1eda70-be5e-42c4-a560-e5a12cee0895 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-ec1eda70-be5e-42c4-a560-e5a12cee0895 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:45.050675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:45.051691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:45.057264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:45.057264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:45.057264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:45.057264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:45.057264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:45.057264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:45.057264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:45.057264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:45.057264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 511/1028] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:45.059801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6efb6a-2645-48e2-9b06-978d2e92c890 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:45.178223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6efb6a-2645-48e2-9b06-978d2e92c890 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:45.206774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542bb9-18ae-40ed-83be-91fc3a839497 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.207060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542bb9-18ae-40ed-83be-91fc3a839497 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.207336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542bb9-18ae-40ed-83be-91fc3a839497 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.208733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 512/1029] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:45.215653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e1552-9103-46c5-bb3f-3786c906ee8b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.215938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e1552-9103-46c5-bb3f-3786c906ee8b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.216210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e1552-9103-46c5-bb3f-3786c906ee8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.221176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75ff46-c7c0-4a40-a89f-e6cf16af49e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:45.222705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75ff46-c7c0-4a40-a89f-e6cf16af49e1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.223204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75ff46-c7c0-4a40-a89f-e6cf16af49e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.223683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75ff46-c7c0-4a40-a89f-e6cf16af49e1 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.224934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75ff46-c7c0-4a40-a89f-e6cf16af49e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:45.225469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75ff46-c7c0-4a40-a89f-e6cf16af49e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:45.227113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75ff46-c7c0-4a40-a89f-e6cf16af49e1 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:45.233834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-0e4e1552-9103-46c5-bb3f-3786c906ee8b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:45.235020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0e4e1552-9103-46c5-bb3f-3786c906ee8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:45.246445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 518/1030] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:45.247356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75ff46-c7c0-4a40-a89f-e6cf16af49e1 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:45.305596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:45.306106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 513/1031] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:45.329731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e1552-9103-46c5-bb3f-3786c906ee8b None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:45.362193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75ff46-c7c0-4a40-a89f-e6cf16af49e1 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:45.398208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30087bb7-61ec-4bbd-beff-0fd6e2e98dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:45.398426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:45.399223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30087bb7-61ec-4bbd-beff-0fd6e2e98dd5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.399547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30087bb7-61ec-4bbd-beff-0fd6e2e98dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.399859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30087bb7-61ec-4bbd-beff-0fd6e2e98dd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.400798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30087bb7-61ec-4bbd-beff-0fd6e2e98dd5 None admin] RBAC: Adding query filter params (role.id=c22095e125674044bf3d2fe3122eeacd, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=928b8b2b510b4c939b37b2c0dfa57060) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:45.401110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30087bb7-61ec-4bbd-beff-0fd6e2e98dd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:45.402551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30087bb7-61ec-4bbd-beff-0fd6e2e98dd5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:45.408545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 519/1032] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:45 2022] GET /identity/v3/role_assignments?user.id=928b8b2b510b4c939b37b2c0dfa57060&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=c22095e125674044bf3d2fe3122eeacd => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:45.449158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e1552-9103-46c5-bb3f-3786c906ee8b None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:45.488432 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1d951-49b0-4387-8cd1-7198ad256525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:45.489259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1d951-49b0-4387-8cd1-7198ad256525 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.489536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1d951-49b0-4387-8cd1-7198ad256525 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.489805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1d951-49b0-4387-8cd1-7198ad256525 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.491056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1d951-49b0-4387-8cd1-7198ad256525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:45.492672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1d951-49b0-4387-8cd1-7198ad256525 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:45.494562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 514/1033] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:45.500260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30087bb7-61ec-4bbd-beff-0fd6e2e98dd5 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:45.584912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381890dd-5a28-4f9b-b4f7-05aaab453e43 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.585455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381890dd-5a28-4f9b-b4f7-05aaab453e43 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.585455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381890dd-5a28-4f9b-b4f7-05aaab453e43 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.587167 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 515/1034] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:45.592406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0da4a0-4ed3-4d22-9c2f-6c6def8b7c88 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.592666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0da4a0-4ed3-4d22-9c2f-6c6def8b7c88 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.592940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0da4a0-4ed3-4d22-9c2f-6c6def8b7c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.610091 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-cb0da4a0-4ed3-4d22-9c2f-6c6def8b7c88 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:45.611302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cb0da4a0-4ed3-4d22-9c2f-6c6def8b7c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:45.616545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30087bb7-61ec-4bbd-beff-0fd6e2e98dd5 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:45.632048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:45.654667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d9af4-660e-4a53-a38d-02dfd330629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:45.655333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d9af4-660e-4a53-a38d-02dfd330629b None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.655616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d9af4-660e-4a53-a38d-02dfd330629b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.655891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d9af4-660e-4a53-a38d-02dfd330629b None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.656647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d9af4-660e-4a53-a38d-02dfd330629b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:45.658067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d9af4-660e-4a53-a38d-02dfd330629b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:45.687016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 520/1035] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:41:45 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:45.698693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 516/1036] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:45.722695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d9af4-660e-4a53-a38d-02dfd330629b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:45.848587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d9af4-660e-4a53-a38d-02dfd330629b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:45.885571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36573444-4812-48d9-bb23-f2a563acfbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:45.886373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36573444-4812-48d9-bb23-f2a563acfbed None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:45.886694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36573444-4812-48d9-bb23-f2a563acfbed None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:45.886999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36573444-4812-48d9-bb23-f2a563acfbed None admin] PATH_INFO: `/v3/services/image` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:45.887625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36573444-4812-48d9-bb23-f2a563acfbed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:45.889038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36573444-4812-48d9-bb23-f2a563acfbed None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-36573444-4812-48d9-bb23-f2a563acfbed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:45.901801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:45.903038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:45.904083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 521/1037] 10.222.0.85 () {56 vars in 1105 bytes} [Tue Jul 19 20:41:45 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:45.910541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0da4a0-4ed3-4d22-9c2f-6c6def8b7c88 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:45.973185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:46.019940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0da4a0-4ed3-4d22-9c2f-6c6def8b7c88 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:46.058621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee62da0-42ba-44c5-b64d-8b91b479fe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:46.059381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee62da0-42ba-44c5-b64d-8b91b479fe94 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:46.059667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee62da0-42ba-44c5-b64d-8b91b479fe94 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:46.059946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee62da0-42ba-44c5-b64d-8b91b479fe94 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:46.060752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee62da0-42ba-44c5-b64d-8b91b479fe94 None admin] RBAC: Adding query filter params (name=image) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:46.061065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee62da0-42ba-44c5-b64d-8b91b479fe94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:46.062448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee62da0-42ba-44c5-b64d-8b91b479fe94 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:46.072940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 517/1038] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:46.074139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36573444-4812-48d9-bb23-f2a563acfbed None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:46.124515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:46.208739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36573444-4812-48d9-bb23-f2a563acfbed None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:46.263056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb9b76-e109-4b14-a544-8992d4e3b091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:46.264709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb9b76-e109-4b14-a544-8992d4e3b091 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:46.265483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb9b76-e109-4b14-a544-8992d4e3b091 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:46.266232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb9b76-e109-4b14-a544-8992d4e3b091 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:46.267824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb9b76-e109-4b14-a544-8992d4e3b091 None admin] RBAC: Adding query filter params (type=image) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:46.268690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb9b76-e109-4b14-a544-8992d4e3b091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:46.271447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb9b76-e109-4b14-a544-8992d4e3b091 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:46.285398 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 522/1039] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:46.292903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee62da0-42ba-44c5-b64d-8b91b479fe94 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:46.415343 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee62da0-42ba-44c5-b64d-8b91b479fe94 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:46.440721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:46.460505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24dd93-4bd1-49b4-83a7-9c4b76463c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:46.461773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24dd93-4bd1-49b4-83a7-9c4b76463c0e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:46.462269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24dd93-4bd1-49b4-83a7-9c4b76463c0e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:46.462803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24dd93-4bd1-49b4-83a7-9c4b76463c0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:46.464180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24dd93-4bd1-49b4-83a7-9c4b76463c0e None admin] RBAC: Adding query filter params (interface=public, service_id=0b17ccc8652f4400aa8affcac2a9d44f, region_id=RegionOne) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:46.465166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24dd93-4bd1-49b4-83a7-9c4b76463c0e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:46.467832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24dd93-4bd1-49b4-83a7-9c4b76463c0e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:46.481053 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb9b76-e109-4b14-a544-8992d4e3b091 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:46.490382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 518/1040] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jul 19 20:41:46 2022] GET /identity/v3/endpoints?service_id=0b17ccc8652f4400aa8affcac2a9d44f&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:46.585908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:46.594781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb9b76-e109-4b14-a544-8992d4e3b091 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:46.631915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31c6fc-4277-40e1-8a49-21209992ef30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:46.632624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31c6fc-4277-40e1-8a49-21209992ef30 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:46.632911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31c6fc-4277-40e1-8a49-21209992ef30 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:46.633187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31c6fc-4277-40e1-8a49-21209992ef30 None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:46.633866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31c6fc-4277-40e1-8a49-21209992ef30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:46.635244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31c6fc-4277-40e1-8a49-21209992ef30 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:46.648645 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 523/1041] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jul 19 20:41:46 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:41:47.780926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cd01a-4797-4741-93ec-8da1141f8dfa None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:47.781567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cd01a-4797-4741-93ec-8da1141f8dfa None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:47.781567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cd01a-4797-4741-93ec-8da1141f8dfa None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:47.782843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 519/1042] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:47.788609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44c19-5d22-473f-80f7-f9ecbfc4ba81 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:47.789176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44c19-5d22-473f-80f7-f9ecbfc4ba81 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:47.789176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d44c19-5d22-473f-80f7-f9ecbfc4ba81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:47.806455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-07d44c19-5d22-473f-80f7-f9ecbfc4ba81 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:47.808254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-07d44c19-5d22-473f-80f7-f9ecbfc4ba81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:47.875444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de75a65-ba15-4c51-9c76-dac039aacfe0 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:47.875786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de75a65-ba15-4c51-9c76-dac039aacfe0 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:47.876006 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de75a65-ba15-4c51-9c76-dac039aacfe0 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:47.877414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 520/1043] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:47.883012 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 524/1044] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:47.883177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919e18f-55f2-4f62-b0a4-9e1adacfa2e2 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:47.883418 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919e18f-55f2-4f62-b0a4-9e1adacfa2e2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:47.883703 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919e18f-55f2-4f62-b0a4-9e1adacfa2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:47.901942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-1919e18f-55f2-4f62-b0a4-9e1adacfa2e2 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:47.903579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1919e18f-55f2-4f62-b0a4-9e1adacfa2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:47.906135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d44c19-5d22-473f-80f7-f9ecbfc4ba81 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:47.977825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 521/1045] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:48.002734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919e18f-55f2-4f62-b0a4-9e1adacfa2e2 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:48.004704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:48.019590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d44c19-5d22-473f-80f7-f9ecbfc4ba81 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:48.058723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c08a8-cf4f-4d60-979c-f52e425b1198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:48.059464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c08a8-cf4f-4d60-979c-f52e425b1198 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:48.059753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c08a8-cf4f-4d60-979c-f52e425b1198 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:48.060059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c08a8-cf4f-4d60-979c-f52e425b1198 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:48.071710 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c08a8-cf4f-4d60-979c-f52e425b1198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-994c08a8-cf4f-4d60-979c-f52e425b1198 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:48.079985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:48.081037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:48.082033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 525/1046] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jul 19 20:41:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:48.087770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c08a8-cf4f-4d60-979c-f52e425b1198 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:48.117203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919e18f-55f2-4f62-b0a4-9e1adacfa2e2 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:48.132859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:48.155350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b376c6-9012-46d6-bced-7c0508f324a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:48.156149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b376c6-9012-46d6-bced-7c0508f324a7 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:48.156477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b376c6-9012-46d6-bced-7c0508f324a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:48.156753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b376c6-9012-46d6-bced-7c0508f324a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:48.157975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b376c6-9012-46d6-bced-7c0508f324a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:48.159480 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b376c6-9012-46d6-bced-7c0508f324a7 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:48.165787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 522/1047] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:48.167200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b376c6-9012-46d6-bced-7c0508f324a7 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:48.200668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c08a8-cf4f-4d60-979c-f52e425b1198 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:48.239642 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90cd4b-2d12-47f8-8474-47603cb249c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:48.240319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90cd4b-2d12-47f8-8474-47603cb249c8 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:48.240637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90cd4b-2d12-47f8-8474-47603cb249c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:48.240921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90cd4b-2d12-47f8-8474-47603cb249c8 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:48.241675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90cd4b-2d12-47f8-8474-47603cb249c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:48.242008 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90cd4b-2d12-47f8-8474-47603cb249c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:48.243360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90cd4b-2d12-47f8-8474-47603cb249c8 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:48.252268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 526/1048] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jul 19 20:41:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:48.257803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90cd4b-2d12-47f8-8474-47603cb249c8 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:48.270768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:48.301669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b376c6-9012-46d6-bced-7c0508f324a7 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:48.340419 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4b113-9fe2-49d8-a122-de469e79739b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:48.341098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4b113-9fe2-49d8-a122-de469e79739b None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:48.341700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4b113-9fe2-49d8-a122-de469e79739b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:48.341813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4b113-9fe2-49d8-a122-de469e79739b None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:48.342697 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4b113-9fe2-49d8-a122-de469e79739b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:48.344225 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4b113-9fe2-49d8-a122-de469e79739b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:48.368186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 523/1049] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:41:48 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:41:48.379961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90cd4b-2d12-47f8-8474-47603cb249c8 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:48.419531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfd79d-9736-42be-97f8-927873492a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:48.420393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfd79d-9736-42be-97f8-927873492a6d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:48.420684 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfd79d-9736-42be-97f8-927873492a6d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:48.420962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfd79d-9736-42be-97f8-927873492a6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:48.434617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfd79d-9736-42be-97f8-927873492a6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:48.436150 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfd79d-9736-42be-97f8-927873492a6d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-cbcfd79d-9736-42be-97f8-927873492a6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:48.455945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:48.457354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:48.458372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:48.458372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:48.458372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:48.458372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:48.458372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 527/1050] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:48.464458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4b113-9fe2-49d8-a122-de469e79739b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:48.511034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:48.579689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4b113-9fe2-49d8-a122-de469e79739b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:48.621452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba3d3b-7745-4623-8d69-cf7d7341af97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:48.622056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba3d3b-7745-4623-8d69-cf7d7341af97 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:48.622346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba3d3b-7745-4623-8d69-cf7d7341af97 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:48.622628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba3d3b-7745-4623-8d69-cf7d7341af97 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:48.623361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba3d3b-7745-4623-8d69-cf7d7341af97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:48.623676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba3d3b-7745-4623-8d69-cf7d7341af97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:48.625149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba3d3b-7745-4623-8d69-cf7d7341af97 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:48.638346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 524/1051] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:48.644615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfd79d-9736-42be-97f8-927873492a6d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:48.699133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33398fc4-0947-46b1-a5b0-1995619cc08e None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:48.699421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33398fc4-0947-46b1-a5b0-1995619cc08e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:48.699700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33398fc4-0947-46b1-a5b0-1995619cc08e None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:48.701376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 525/1052] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:48.706404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b85a1-46dd-476f-a86b-e96239979335 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:48.706896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b85a1-46dd-476f-a86b-e96239979335 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:48.707327 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b85a1-46dd-476f-a86b-e96239979335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:48.716792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:48.726742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-945b85a1-46dd-476f-a86b-e96239979335 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:48.727988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-945b85a1-46dd-476f-a86b-e96239979335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:48.763545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfd79d-9736-42be-97f8-927873492a6d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:48.800725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100757c-79c7-4aa3-9665-5c30290c1191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:48.801612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100757c-79c7-4aa3-9665-5c30290c1191 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:48.801866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100757c-79c7-4aa3-9665-5c30290c1191 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:48.802147 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100757c-79c7-4aa3-9665-5c30290c1191 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:48.808066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 526/1053] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:48.811368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100757c-79c7-4aa3-9665-5c30290c1191 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:48.812863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100757c-79c7-4aa3-9665-5c30290c1191 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-3100757c-79c7-4aa3-9665-5c30290c1191 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:48.827813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:48.829093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:48.830064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:48.830064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:48.830064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:48.830064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:48.830064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:48.830064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:48.830064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:48.830064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:48.830064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:41:48.830064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:48.831928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b85a1-46dd-476f-a86b-e96239979335 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:48.832667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 528/1054] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jul 19 20:41:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:48.834827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100757c-79c7-4aa3-9665-5c30290c1191 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:48.860558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:48.943407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100757c-79c7-4aa3-9665-5c30290c1191 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:48.943759 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b85a1-46dd-476f-a86b-e96239979335 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:48.968049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:48.981319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753af91-d205-49de-bde5-e87872b527c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:48.981999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753af91-d205-49de-bde5-e87872b527c5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:48.982297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753af91-d205-49de-bde5-e87872b527c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:48.982557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753af91-d205-49de-bde5-e87872b527c5 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:48.982953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce3c63-053e-4483-a2ea-055040c009e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:48.983402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753af91-d205-49de-bde5-e87872b527c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:48.983715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753af91-d205-49de-bde5-e87872b527c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:48.983809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3c63-053e-4483-a2ea-055040c009e2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:48.984015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3c63-053e-4483-a2ea-055040c009e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:48.984290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3c63-053e-4483-a2ea-055040c009e2 None admin] PATH_INFO: `/v3/services/image` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:48.984978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce3c63-053e-4483-a2ea-055040c009e2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:48.985162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753af91-d205-49de-bde5-e87872b527c5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:48.986332 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce3c63-053e-4483-a2ea-055040c009e2 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-30ce3c63-053e-4483-a2ea-055040c009e2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:48.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:49.000257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:49.001163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 527/1055] 10.222.0.85 () {56 vars in 1105 bytes} [Tue Jul 19 20:41:48 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:49.011529 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce3c63-053e-4483-a2ea-055040c009e2 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:49.011718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 529/1056] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jul 19 20:41:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:49.012456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753af91-d205-49de-bde5-e87872b527c5 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:49.107629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:49.119010 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce3c63-053e-4483-a2ea-055040c009e2 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:49.119641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753af91-d205-49de-bde5-e87872b527c5 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:49.154468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ed41b-6c9b-4cb2-b2d7-8fde10b9ff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:49.154777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8954740-639d-4f23-9698-9d54ebeb89d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:49.155544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8954740-639d-4f23-9698-9d54ebeb89d4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:49.155836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8954740-639d-4f23-9698-9d54ebeb89d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:49.156014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ed41b-6c9b-4cb2-b2d7-8fde10b9ff2c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:49.156120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8954740-639d-4f23-9698-9d54ebeb89d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:49.156631 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ed41b-6c9b-4cb2-b2d7-8fde10b9ff2c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:49.157073 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ed41b-6c9b-4cb2-b2d7-8fde10b9ff2c None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:49.157953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ed41b-6c9b-4cb2-b2d7-8fde10b9ff2c None admin] RBAC: Adding query filter params (name=image) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:49.158462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ed41b-6c9b-4cb2-b2d7-8fde10b9ff2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:49.160191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ed41b-6c9b-4cb2-b2d7-8fde10b9ff2c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:49.167091 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 528/1057] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:49.168504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8954740-639d-4f23-9698-9d54ebeb89d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:49.170084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8954740-639d-4f23-9698-9d54ebeb89d4 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:49.173028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ed41b-6c9b-4cb2-b2d7-8fde10b9ff2c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-a8954740-639d-4f23-9698-9d54ebeb89d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:49.197980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:49.199095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:49.200019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:49.200019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:49.200019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:49.200019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:49.205031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 530/1058] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:49.206249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8954740-639d-4f23-9698-9d54ebeb89d4 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:49.250419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:49.303074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ed41b-6c9b-4cb2-b2d7-8fde10b9ff2c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:49.341199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92eb390-d430-4245-9f3b-20be534cbdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:49.341444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8954740-639d-4f23-9698-9d54ebeb89d4 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:49.341876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eb390-d430-4245-9f3b-20be534cbdad None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:49.342333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eb390-d430-4245-9f3b-20be534cbdad None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:49.342720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eb390-d430-4245-9f3b-20be534cbdad None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:49.343324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92eb390-d430-4245-9f3b-20be534cbdad None admin] RBAC: Adding query filter params (type=image) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:49.343634 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92eb390-d430-4245-9f3b-20be534cbdad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:49.345047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92eb390-d430-4245-9f3b-20be534cbdad None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:49.350804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 529/1059] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:49.356542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92eb390-d430-4245-9f3b-20be534cbdad None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:49.385698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:49.391778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40439925-a0b2-49fe-82a3-97e4be3eb191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:49.392472 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40439925-a0b2-49fe-82a3-97e4be3eb191 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:49.392776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40439925-a0b2-49fe-82a3-97e4be3eb191 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:49.393050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40439925-a0b2-49fe-82a3-97e4be3eb191 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:49.393771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40439925-a0b2-49fe-82a3-97e4be3eb191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:49.394076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40439925-a0b2-49fe-82a3-97e4be3eb191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:49.395430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40439925-a0b2-49fe-82a3-97e4be3eb191 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:49.406897 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 531/1060] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:49.412960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40439925-a0b2-49fe-82a3-97e4be3eb191 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:49.471668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92eb390-d430-4245-9f3b-20be534cbdad None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:49.510492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac25bce-3df0-4bcd-9b7c-63519e3a4f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:49.510733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:49.511159 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac25bce-3df0-4bcd-9b7c-63519e3a4f13 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:49.511446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac25bce-3df0-4bcd-9b7c-63519e3a4f13 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:49.511722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac25bce-3df0-4bcd-9b7c-63519e3a4f13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:49.512572 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac25bce-3df0-4bcd-9b7c-63519e3a4f13 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:49.513944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac25bce-3df0-4bcd-9b7c-63519e3a4f13 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:49.521348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40439925-a0b2-49fe-82a3-97e4be3eb191 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:49.533618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 530/1061] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jul 19 20:41:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:41:49.570568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388013f-84a3-4b78-bfc9-b0d9ce44cba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:49.571900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388013f-84a3-4b78-bfc9-b0d9ce44cba0 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:49.572484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388013f-84a3-4b78-bfc9-b0d9ce44cba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:49.573014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388013f-84a3-4b78-bfc9-b0d9ce44cba0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:49.581406 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388013f-84a3-4b78-bfc9-b0d9ce44cba0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:49.583201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388013f-84a3-4b78-bfc9-b0d9ce44cba0 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-1388013f-84a3-4b78-bfc9-b0d9ce44cba0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:49.606528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:49.607565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:49.608646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:49.608646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:49.608646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:49.608646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:49.608646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:49.608646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:49.608646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:49.608646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:49.615188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 532/1062] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:49.616486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac25bce-3df0-4bcd-9b7c-63519e3a4f13 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:49.705100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:49.733979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac25bce-3df0-4bcd-9b7c-63519e3a4f13 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:49.770591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3715b-fbef-4ffb-ad34-de02cc83dd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:49.771257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3715b-fbef-4ffb-ad34-de02cc83dd16 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:49.771537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3715b-fbef-4ffb-ad34-de02cc83dd16 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:49.771809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3715b-fbef-4ffb-ad34-de02cc83dd16 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:49.772641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3715b-fbef-4ffb-ad34-de02cc83dd16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:49.772954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3715b-fbef-4ffb-ad34-de02cc83dd16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:49.774388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3715b-fbef-4ffb-ad34-de02cc83dd16 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:49.787650 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 531/1063] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:49.793438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388013f-84a3-4b78-bfc9-b0d9ce44cba0 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:49.900199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388013f-84a3-4b78-bfc9-b0d9ce44cba0 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:49.935083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c9113-79b2-45b3-a7df-6b8471a61ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:49.935979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c9113-79b2-45b3-a7df-6b8471a61ede None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:49.936261 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c9113-79b2-45b3-a7df-6b8471a61ede None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:49.936568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c9113-79b2-45b3-a7df-6b8471a61ede None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/928b8b2b510b4c939b37b2c0dfa57060/roles/c22095e125674044bf3d2fe3122eeacd` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:49.938764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c9113-79b2-45b3-a7df-6b8471a61ede None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=928b8b2b510b4c939b37b2c0dfa57060, role_id=c22095e125674044bf3d2fe3122eeacd)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:49.940203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c9113-79b2-45b3-a7df-6b8471a61ede None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:49.958895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 533/1064] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:41:49 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/928b8b2b510b4c939b37b2c0dfa57060/roles/c22095e125674044bf3d2fe3122eeacd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:41:50.549946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70738e48-8441-46e0-8205-c229dac0b33f None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:50.550461 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70738e48-8441-46e0-8205-c229dac0b33f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:50.550461 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70738e48-8441-46e0-8205-c229dac0b33f None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:50.551906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 532/1065] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:50.557749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150641f0-a3cc-4e68-a74d-93039d46951f None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:50.558039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150641f0-a3cc-4e68-a74d-93039d46951f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:50.558312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150641f0-a3cc-4e68-a74d-93039d46951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:50.575118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-150641f0-a3cc-4e68-a74d-93039d46951f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:50.576304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-150641f0-a3cc-4e68-a74d-93039d46951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:50.578353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:50.647757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 534/1066] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:50.671899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70738e48-8441-46e0-8205-c229dac0b33f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:50.778265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70738e48-8441-46e0-8205-c229dac0b33f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:50.811687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918045a-ecb9-4ed5-885e-aa80c0da0498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:50.812414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918045a-ecb9-4ed5-885e-aa80c0da0498 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:50.812699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918045a-ecb9-4ed5-885e-aa80c0da0498 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:50.812974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918045a-ecb9-4ed5-885e-aa80c0da0498 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:50.815612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:50.825319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918045a-ecb9-4ed5-885e-aa80c0da0498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-7918045a-ecb9-4ed5-885e-aa80c0da0498 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:50.833651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:50.834586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:50.835436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:50.842662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 533/1067] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:41:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:50.844226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150641f0-a3cc-4e68-a74d-93039d46951f None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:50.947664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150641f0-a3cc-4e68-a74d-93039d46951f None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:50.987425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5892ee-45d5-45ca-a4cf-b283da88e655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:50.988097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5892ee-45d5-45ca-a4cf-b283da88e655 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:50.988402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5892ee-45d5-45ca-a4cf-b283da88e655 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:50.988679 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5892ee-45d5-45ca-a4cf-b283da88e655 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:50.989436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5892ee-45d5-45ca-a4cf-b283da88e655 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:50.989743 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5892ee-45d5-45ca-a4cf-b283da88e655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:50.991108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5892ee-45d5-45ca-a4cf-b283da88e655 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:50.996938 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:51.005806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 535/1068] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:41:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:51.007423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918045a-ecb9-4ed5-885e-aa80c0da0498 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:51.115930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918045a-ecb9-4ed5-885e-aa80c0da0498 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:51.152122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025fa60-1ee1-4c1e-add8-1bf93c964a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:51.153377 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025fa60-1ee1-4c1e-add8-1bf93c964a88 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:51.153871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025fa60-1ee1-4c1e-add8-1bf93c964a88 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:51.154388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025fa60-1ee1-4c1e-add8-1bf93c964a88 None admin] PATH_INFO: `/v3/users/b3c975f2cbbb460fad47c5ab1daab49e` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:51.168542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025fa60-1ee1-4c1e-add8-1bf93c964a88 None admin] RBAC: Authorizing `identity:get_user(user_id=b3c975f2cbbb460fad47c5ab1daab49e)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:51.170270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025fa60-1ee1-4c1e-add8-1bf93c964a88 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:51.176508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 534/1069] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:41:51 2022] GET /identity/v3/users/b3c975f2cbbb460fad47c5ab1daab49e => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:51.178903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5892ee-45d5-45ca-a4cf-b283da88e655 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:51.291251 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5892ee-45d5-45ca-a4cf-b283da88e655 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:51.327220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083aa41-f9ec-4b6b-abb5-dac0a60a930b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:51.327872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083aa41-f9ec-4b6b-abb5-dac0a60a930b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:51.328158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083aa41-f9ec-4b6b-abb5-dac0a60a930b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:51.328465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083aa41-f9ec-4b6b-abb5-dac0a60a930b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:51.410782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:51.414325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083aa41-f9ec-4b6b-abb5-dac0a60a930b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:51.416462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083aa41-f9ec-4b6b-abb5-dac0a60a930b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:51.522736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 536/1070] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 20:41:51.523973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025fa60-1ee1-4c1e-add8-1bf93c964a88 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:51.582814 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:51.630989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025fa60-1ee1-4c1e-add8-1bf93c964a88 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:51.633788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a914e-38da-44e0-bab7-495975f54dc1 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:51.634080 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a914e-38da-44e0-bab7-495975f54dc1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:51.634360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a914e-38da-44e0-bab7-495975f54dc1 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:51.635877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 537/1071] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:51.640730 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da4e43-e6da-4076-97b8-2b80b3416c5a None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:51.641025 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da4e43-e6da-4076-97b8-2b80b3416c5a None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:51.641310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da4e43-e6da-4076-97b8-2b80b3416c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:51.663048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-b1da4e43-e6da-4076-97b8-2b80b3416c5a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:51.664169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b1da4e43-e6da-4076-97b8-2b80b3416c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:51.669034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249ab00-64c3-4b19-8ab7-00a4e0f4249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:51.670278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249ab00-64c3-4b19-8ab7-00a4e0f4249d None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:51.670684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249ab00-64c3-4b19-8ab7-00a4e0f4249d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:51.671085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249ab00-64c3-4b19-8ab7-00a4e0f4249d None admin] PATH_INFO: `/v3/projects/cca349ccce3744ca801e4749d66ace8c` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:51.672989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249ab00-64c3-4b19-8ab7-00a4e0f4249d None admin] RBAC: Authorizing `identity:get_project(project_id=cca349ccce3744ca801e4749d66ace8c)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:51.675054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249ab00-64c3-4b19-8ab7-00a4e0f4249d None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:51.683172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 535/1072] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:41:51 2022] GET /identity/v3/projects/cca349ccce3744ca801e4749d66ace8c => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:51.684232 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249ab00-64c3-4b19-8ab7-00a4e0f4249d None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:51.756907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 538/1073] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:51.779170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da4e43-e6da-4076-97b8-2b80b3416c5a None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:51.816265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:51.818533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249ab00-64c3-4b19-8ab7-00a4e0f4249d None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:51.850604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:51.870261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061821d-45ef-49b6-969e-b2405678e489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:51.871013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061821d-45ef-49b6-969e-b2405678e489 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:51.871387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061821d-45ef-49b6-969e-b2405678e489 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:51.871684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061821d-45ef-49b6-969e-b2405678e489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:51.872552 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061821d-45ef-49b6-969e-b2405678e489 None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=cca349ccce3744ca801e4749d66ace8c, user.id=b3c975f2cbbb460fad47c5ab1daab49e) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:51.872870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061821d-45ef-49b6-969e-b2405678e489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:51.874313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061821d-45ef-49b6-969e-b2405678e489 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:51.888369 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 536/1074] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:51 2022] GET /identity/v3/role_assignments?user.id=b3c975f2cbbb460fad47c5ab1daab49e&scope.project.id=cca349ccce3744ca801e4749d66ace8c&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:51.911189 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da4e43-e6da-4076-97b8-2b80b3416c5a None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:51.957751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83205f3-ca4c-4fbf-94e7-4adf21efb155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:51.958573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83205f3-ca4c-4fbf-94e7-4adf21efb155 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:51.958882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83205f3-ca4c-4fbf-94e7-4adf21efb155 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:51.959381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83205f3-ca4c-4fbf-94e7-4adf21efb155 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:52.018181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6c1d5-5b0d-4d6f-a29f-4734c2782e21 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:52.019086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6c1d5-5b0d-4d6f-a29f-4734c2782e21 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:52.019292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6c1d5-5b0d-4d6f-a29f-4734c2782e21 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:52.020819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 537/1075] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:52.026902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1fb98-e41a-4ad5-a86c-8ddc8e5fc8f4 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:52.027266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1fb98-e41a-4ad5-a86c-8ddc8e5fc8f4 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:52.027452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1fb98-e41a-4ad5-a86c-8ddc8e5fc8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:52.048838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-49c1fb98-e41a-4ad5-a86c-8ddc8e5fc8f4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:52.050135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-49c1fb98-e41a-4ad5-a86c-8ddc8e5fc8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:52.053228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83205f3-ca4c-4fbf-94e7-4adf21efb155 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:52.054763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83205f3-ca4c-4fbf-94e7-4adf21efb155 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:52.088584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:52.130865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 538/1076] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:52.154810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1fb98-e41a-4ad5-a86c-8ddc8e5fc8f4 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:52.169661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 539/1077] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 20:41:52.175263 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83205f3-ca4c-4fbf-94e7-4adf21efb155 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:52.282669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83205f3-ca4c-4fbf-94e7-4adf21efb155 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:52.283055 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1fb98-e41a-4ad5-a86c-8ddc8e5fc8f4 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:52.302519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:52.319999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:52.326598 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2ab0a-6c9a-4af5-aedc-8c0625e5efac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:52.326939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a59f05-2fda-4308-991f-bc653714cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:52.327496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ab0a-6c9a-4af5-aedc-8c0625e5efac None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:52.327671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59f05-2fda-4308-991f-bc653714cd45 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:52.327775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ab0a-6c9a-4af5-aedc-8c0625e5efac None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:52.327927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59f05-2fda-4308-991f-bc653714cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:52.328029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ab0a-6c9a-4af5-aedc-8c0625e5efac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:52.328198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59f05-2fda-4308-991f-bc653714cd45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:52.341115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2ab0a-6c9a-4af5-aedc-8c0625e5efac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:52.341942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a59f05-2fda-4308-991f-bc653714cd45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:52.343509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a59f05-2fda-4308-991f-bc653714cd45 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-13e2ab0a-6c9a-4af5-aedc-8c0625e5efac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:52.349656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:52.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:52.352062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 539/1078] 10.222.0.85 () {56 vars in 1120 bytes} [Tue Jul 19 20:41:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:52.359017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2ab0a-6c9a-4af5-aedc-8c0625e5efac None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-e7a59f05-2fda-4308-991f-bc653714cd45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:52.371457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:52.372812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:52.374015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:52.374015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:52.374015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:52.374015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:52.374015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 540/1079] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:52.382672 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a59f05-2fda-4308-991f-bc653714cd45 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:52.476257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2ab0a-6c9a-4af5-aedc-8c0625e5efac None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:52.495679 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a59f05-2fda-4308-991f-bc653714cd45 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:52.513365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c8942-310f-4122-85c8-58b77b5326a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:52.514039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c8942-310f-4122-85c8-58b77b5326a6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:52.514330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c8942-310f-4122-85c8-58b77b5326a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:52.514593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c8942-310f-4122-85c8-58b77b5326a6 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:52.515464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c8942-310f-4122-85c8-58b77b5326a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:52.515667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c8942-310f-4122-85c8-58b77b5326a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:52.517061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c8942-310f-4122-85c8-58b77b5326a6 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:52.527412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 540/1080] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jul 19 20:41:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:52.532969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c8942-310f-4122-85c8-58b77b5326a6 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:52.535484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bff86-70c4-4d1f-8733-4c2ad81f73fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:52.536161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bff86-70c4-4d1f-8733-4c2ad81f73fe None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:52.536484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bff86-70c4-4d1f-8733-4c2ad81f73fe None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:52.536765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bff86-70c4-4d1f-8733-4c2ad81f73fe None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:52.537515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bff86-70c4-4d1f-8733-4c2ad81f73fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:52.537830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bff86-70c4-4d1f-8733-4c2ad81f73fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:52.539227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bff86-70c4-4d1f-8733-4c2ad81f73fe None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:52.560364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 541/1081] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:52.589119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:52.666184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c8942-310f-4122-85c8-58b77b5326a6 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:52.709750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379610c9-4e91-45dc-a8a2-4fbc15cf2270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:52.710561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379610c9-4e91-45dc-a8a2-4fbc15cf2270 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:52.710846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379610c9-4e91-45dc-a8a2-4fbc15cf2270 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:52.711124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379610c9-4e91-45dc-a8a2-4fbc15cf2270 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:52.723251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379610c9-4e91-45dc-a8a2-4fbc15cf2270 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:52.724982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379610c9-4e91-45dc-a8a2-4fbc15cf2270 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-379610c9-4e91-45dc-a8a2-4fbc15cf2270 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:52.746335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:52.747468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:52.748846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:52.748846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:52.748846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:52.748846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:52.748846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 541/1082] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:52.755512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bff86-70c4-4d1f-8733-4c2ad81f73fe None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:52.805471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:52.879660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bff86-70c4-4d1f-8733-4c2ad81f73fe None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:52.915150 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b91a06-840b-439e-a683-117a07358081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:52.915899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91a06-840b-439e-a683-117a07358081 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:52.916183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91a06-840b-439e-a683-117a07358081 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:52.916501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91a06-840b-439e-a683-117a07358081 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:52.917258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b91a06-840b-439e-a683-117a07358081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:52.917568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b91a06-840b-439e-a683-117a07358081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:52.918945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b91a06-840b-439e-a683-117a07358081 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:52.930658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 542/1083] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:52.936729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379610c9-4e91-45dc-a8a2-4fbc15cf2270 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:53.026829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:53.037808 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379610c9-4e91-45dc-a8a2-4fbc15cf2270 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:53.074627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df775000-6857-42b0-ab7b-7d853ddf025c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:53.075417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df775000-6857-42b0-ab7b-7d853ddf025c None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:53.075698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df775000-6857-42b0-ab7b-7d853ddf025c None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:53.075974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df775000-6857-42b0-ab7b-7d853ddf025c None admin] PATH_INFO: `/v3/users/nova` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:53.083836 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df775000-6857-42b0-ab7b-7d853ddf025c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:53.085294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df775000-6857-42b0-ab7b-7d853ddf025c None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-df775000-6857-42b0-ab7b-7d853ddf025c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:41:53.100157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:53.101362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:53.102620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:53.102620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:53.102620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:41:53.102620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:41:53.102620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:41:53.102620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:41:53.102620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:41:53.102620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:41:53.102620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 20:41:53.102620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:53.104472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 542/1084] 10.222.0.85 () {56 vars in 1093 bytes} [Tue Jul 19 20:41:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:53.107064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b91a06-840b-439e-a683-117a07358081 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:53.208652 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b91a06-840b-439e-a683-117a07358081 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:53.210210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:53.245018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a831ee-5ff8-4e15-9f89-b15e04d763f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:53.245691 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a831ee-5ff8-4e15-9f89-b15e04d763f2 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:53.245975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a831ee-5ff8-4e15-9f89-b15e04d763f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:53.246248 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a831ee-5ff8-4e15-9f89-b15e04d763f2 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:53.247084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a831ee-5ff8-4e15-9f89-b15e04d763f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:53.247393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a831ee-5ff8-4e15-9f89-b15e04d763f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:53.248838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a831ee-5ff8-4e15-9f89-b15e04d763f2 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:53.266741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 543/1085] 10.222.0.85 () {56 vars in 1133 bytes} [Tue Jul 19 20:41:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:53.273084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df775000-6857-42b0-ab7b-7d853ddf025c None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:53.374483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df775000-6857-42b0-ab7b-7d853ddf025c None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:53.419800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9504e5-1151-4c64-9ddf-284a9ea079d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:53.420484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9504e5-1151-4c64-9ddf-284a9ea079d9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:53.420771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9504e5-1151-4c64-9ddf-284a9ea079d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:53.421050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9504e5-1151-4c64-9ddf-284a9ea079d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:53.442330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:53.492292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9504e5-1151-4c64-9ddf-284a9ea079d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:53.493774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9504e5-1151-4c64-9ddf-284a9ea079d9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:53.596299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 543/1086] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:41:53.601830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a831ee-5ff8-4e15-9f89-b15e04d763f2 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:53.704097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a831ee-5ff8-4e15-9f89-b15e04d763f2 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:53.740442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7ae9f-4f66-403c-ad67-32661c54f690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:53.741236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7ae9f-4f66-403c-ad67-32661c54f690 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:53.741582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7ae9f-4f66-403c-ad67-32661c54f690 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:53.741926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7ae9f-4f66-403c-ad67-32661c54f690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:53.754241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7ae9f-4f66-403c-ad67-32661c54f690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:53.755880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7ae9f-4f66-403c-ad67-32661c54f690 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-81f7ae9f-4f66-403c-ad67-32661c54f690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:41:53.774524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:53.775512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:53.776908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:41:53.776908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:41:53.776908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:41:53.776908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:53.776908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 544/1087] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:53.781623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9504e5-1151-4c64-9ddf-284a9ea079d9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:53.832392 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:53.890259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9504e5-1151-4c64-9ddf-284a9ea079d9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:53.924436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82349278-e828-4c39-bbe4-f271bc56d566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:53.925727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349278-e828-4c39-bbe4-f271bc56d566 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:53.926051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349278-e828-4c39-bbe4-f271bc56d566 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:53.926370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349278-e828-4c39-bbe4-f271bc56d566 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:53.927206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82349278-e828-4c39-bbe4-f271bc56d566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:53.927565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82349278-e828-4c39-bbe4-f271bc56d566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:53.929108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82349278-e828-4c39-bbe4-f271bc56d566 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:53.941977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 544/1088] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:53.947681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7ae9f-4f66-403c-ad67-32661c54f690 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:53.962774 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:54.049586 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7ae9f-4f66-403c-ad67-32661c54f690 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:54.063757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fc9a3-fe09-4171-a1ab-23bd397f7093 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:54.064046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fc9a3-fe09-4171-a1ab-23bd397f7093 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:54.064362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fc9a3-fe09-4171-a1ab-23bd397f7093 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:54.065798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 545/1089] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:41:54.071676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355111d-2cbf-485e-bf8d-3ba946f72e6a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:54.071961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355111d-2cbf-485e-bf8d-3ba946f72e6a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:54.072247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355111d-2cbf-485e-bf8d-3ba946f72e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:54.090876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-6355111d-2cbf-485e-bf8d-3ba946f72e6a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:54.092092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6355111d-2cbf-485e-bf8d-3ba946f72e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:54.096131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f9ad7-68f4-40d2-bc2c-202cc8db46b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:54.097479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f9ad7-68f4-40d2-bc2c-202cc8db46b7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:54.097758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f9ad7-68f4-40d2-bc2c-202cc8db46b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:54.098029 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f9ad7-68f4-40d2-bc2c-202cc8db46b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:54.105340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f9ad7-68f4-40d2-bc2c-202cc8db46b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:54.106773 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f9ad7-68f4-40d2-bc2c-202cc8db46b7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-ab5f9ad7-68f4-40d2-bc2c-202cc8db46b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:41:54.122373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:54.123797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:54.124820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:41:54.124820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:41:54.124820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:41:54.124820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:41:54.124820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:41:54.124820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:41:54.124820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:41:54.124820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:41:54.124820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 545/1090] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:41:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:54.133932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f9ad7-68f4-40d2-bc2c-202cc8db46b7 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:54.177156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 546/1091] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:41:54.200179 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355111d-2cbf-485e-bf8d-3ba946f72e6a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:54.210519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:54.241915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f9ad7-68f4-40d2-bc2c-202cc8db46b7 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:54.244211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:54.278231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2f1e1-bf3c-49d7-b186-18a460e83e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:54.278924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2f1e1-bf3c-49d7-b186-18a460e83e2a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:54.279213 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2f1e1-bf3c-49d7-b186-18a460e83e2a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:54.279495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2f1e1-bf3c-49d7-b186-18a460e83e2a None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:54.280397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2f1e1-bf3c-49d7-b186-18a460e83e2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:54.280749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2f1e1-bf3c-49d7-b186-18a460e83e2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:54.282287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2f1e1-bf3c-49d7-b186-18a460e83e2a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:54.297239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 546/1092] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:41:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:54.302454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355111d-2cbf-485e-bf8d-3ba946f72e6a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:54.303653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2f1e1-bf3c-49d7-b186-18a460e83e2a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:54.338913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7c2bf-3a59-4e10-a3e4-fda47bea20c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:54.339763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7c2bf-3a59-4e10-a3e4-fda47bea20c9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:54.340051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7c2bf-3a59-4e10-a3e4-fda47bea20c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:54.340351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7c2bf-3a59-4e10-a3e4-fda47bea20c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:54.351975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7c2bf-3a59-4e10-a3e4-fda47bea20c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-beb7c2bf-3a59-4e10-a3e4-fda47bea20c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:54.359685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:54.360763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:54.361842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 547/1093] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:41:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:41:54.367104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7c2bf-3a59-4e10-a3e4-fda47bea20c9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:54.407311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2f1e1-bf3c-49d7-b186-18a460e83e2a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:54.449032 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:54.455013 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32669405-dfd0-4e2a-8ad8-4469808da6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:54.455690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32669405-dfd0-4e2a-8ad8-4469808da6dd None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:54.456502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32669405-dfd0-4e2a-8ad8-4469808da6dd None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:54.456808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32669405-dfd0-4e2a-8ad8-4469808da6dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:54.457638 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32669405-dfd0-4e2a-8ad8-4469808da6dd None admin] RBAC: Adding query filter params (role.id=c22095e125674044bf3d2fe3122eeacd, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=928b8b2b510b4c939b37b2c0dfa57060) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:54.457954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32669405-dfd0-4e2a-8ad8-4469808da6dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:54.459394 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32669405-dfd0-4e2a-8ad8-4469808da6dd None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:54.466326 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 547/1094] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:54 2022] GET /identity/v3/role_assignments?user.id=928b8b2b510b4c939b37b2c0dfa57060&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=c22095e125674044bf3d2fe3122eeacd => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:54.475718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:54.479039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7c2bf-3a59-4e10-a3e4-fda47bea20c9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:54.519704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c23e4-1a2a-40ec-98c6-6d215d692280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:54.520465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c23e4-1a2a-40ec-98c6-6d215d692280 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:54.520878 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c23e4-1a2a-40ec-98c6-6d215d692280 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:54.521281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c23e4-1a2a-40ec-98c6-6d215d692280 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:54.522438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c23e4-1a2a-40ec-98c6-6d215d692280 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:54.522756 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c23e4-1a2a-40ec-98c6-6d215d692280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:54.524243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c23e4-1a2a-40ec-98c6-6d215d692280 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:54.534543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 548/1095] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:41:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:54.540705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32669405-dfd0-4e2a-8ad8-4469808da6dd None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:54.664876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32669405-dfd0-4e2a-8ad8-4469808da6dd None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:54.702854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458e972-338e-4c21-8d19-a8554692c2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:54.703733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458e972-338e-4c21-8d19-a8554692c2f2 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:54.704022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458e972-338e-4c21-8d19-a8554692c2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:54.704300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458e972-338e-4c21-8d19-a8554692c2f2 None admin] PATH_INFO: `/v3/users/b3c975f2cbbb460fad47c5ab1daab49e` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:54.705548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458e972-338e-4c21-8d19-a8554692c2f2 None admin] RBAC: Authorizing `identity:get_user(user_id=b3c975f2cbbb460fad47c5ab1daab49e)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:54.707748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458e972-338e-4c21-8d19-a8554692c2f2 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:54.709460 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 548/1096] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:41:54 2022] GET /identity/v3/users/b3c975f2cbbb460fad47c5ab1daab49e => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:54.715546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c23e4-1a2a-40ec-98c6-6d215d692280 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:54.833748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c23e4-1a2a-40ec-98c6-6d215d692280 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:54.869726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394f4e1-b90f-4b91-b7a9-c6882fea6316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:54.870585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394f4e1-b90f-4b91-b7a9-c6882fea6316 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:54.870874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394f4e1-b90f-4b91-b7a9-c6882fea6316 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:54.871141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394f4e1-b90f-4b91-b7a9-c6882fea6316 None admin] PATH_INFO: `/v3/projects/cca349ccce3744ca801e4749d66ace8c` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:54.874096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394f4e1-b90f-4b91-b7a9-c6882fea6316 None admin] RBAC: Authorizing `identity:get_project(project_id=cca349ccce3744ca801e4749d66ace8c)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:54.875550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394f4e1-b90f-4b91-b7a9-c6882fea6316 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:54.877468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 549/1097] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:41:54 2022] GET /identity/v3/projects/cca349ccce3744ca801e4749d66ace8c => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:54.884137 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458e972-338e-4c21-8d19-a8554692c2f2 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:54.960804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:55.008481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458e972-338e-4c21-8d19-a8554692c2f2 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:55.043634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1a0d9-37ea-4fb3-a0eb-2d6e0cd2c267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:55.044555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a0d9-37ea-4fb3-a0eb-2d6e0cd2c267 None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:55.044837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a0d9-37ea-4fb3-a0eb-2d6e0cd2c267 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:55.045111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1a0d9-37ea-4fb3-a0eb-2d6e0cd2c267 None admin] PATH_INFO: `/v3/projects/cca349ccce3744ca801e4749d66ace8c/users/b3c975f2cbbb460fad47c5ab1daab49e/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:55.051226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1a0d9-37ea-4fb3-a0eb-2d6e0cd2c267 None admin] RBAC: Authorizing `identity:create_grant(project_id=cca349ccce3744ca801e4749d66ace8c, user_id=b3c975f2cbbb460fad47c5ab1daab49e, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:55.053940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1a0d9-37ea-4fb3-a0eb-2d6e0cd2c267 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:55.069104 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 549/1098] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:41:54 2022] PUT /identity/v3/projects/cca349ccce3744ca801e4749d66ace8c/users/b3c975f2cbbb460fad47c5ab1daab49e/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:41:57.111661 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5f274-b84b-48c6-a76f-bf55a23792b6 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:57.112288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5f274-b84b-48c6-a76f-bf55a23792b6 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:57.112916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5f274-b84b-48c6-a76f-bf55a23792b6 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:57.114206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 550/1099] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:41:57.119960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d90b30-f505-41f3-8c97-e1ec2580cb1e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:57.120556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d90b30-f505-41f3-8c97-e1ec2580cb1e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:57.120556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d90b30-f505-41f3-8c97-e1ec2580cb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:57.138361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c6d90b30-f505-41f3-8c97-e1ec2580cb1e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:41:57.139613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c6d90b30-f505-41f3-8c97-e1ec2580cb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:41:57.213129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 550/1100] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:41:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:41:57.236298 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5f274-b84b-48c6-a76f-bf55a23792b6 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:57.269366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:57.345493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5f274-b84b-48c6-a76f-bf55a23792b6 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:57.379366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f826d2e-0a4e-4373-b7d4-fdc30da4fb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:57.380074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f826d2e-0a4e-4373-b7d4-fdc30da4fb16 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:57.380387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f826d2e-0a4e-4373-b7d4-fdc30da4fb16 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:57.380794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f826d2e-0a4e-4373-b7d4-fdc30da4fb16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:57.390809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f826d2e-0a4e-4373-b7d4-fdc30da4fb16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-3f826d2e-0a4e-4373-b7d4-fdc30da4fb16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:41:57.398980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:41:57.400105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:41:57.401084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 551/1101] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:41:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:41:57.406361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d90b30-f505-41f3-8c97-e1ec2580cb1e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:57.460870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:57.509042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d90b30-f505-41f3-8c97-e1ec2580cb1e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:57.542230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f1e60-b9ad-4626-9f6c-8ac56caf33a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:57.542899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f1e60-b9ad-4626-9f6c-8ac56caf33a1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:57.543179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f1e60-b9ad-4626-9f6c-8ac56caf33a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:57.543450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f1e60-b9ad-4626-9f6c-8ac56caf33a1 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:57.544192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f1e60-b9ad-4626-9f6c-8ac56caf33a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:57.544526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f1e60-b9ad-4626-9f6c-8ac56caf33a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:57.545881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f1e60-b9ad-4626-9f6c-8ac56caf33a1 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:57.559082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 551/1102] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:41:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:57.565106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f826d2e-0a4e-4373-b7d4-fdc30da4fb16 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:57.665981 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:57.675945 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f826d2e-0a4e-4373-b7d4-fdc30da4fb16 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:57.710561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43748fd2-22eb-411a-b26b-161e840d414e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:57.711414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43748fd2-22eb-411a-b26b-161e840d414e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:57.711694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43748fd2-22eb-411a-b26b-161e840d414e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:57.711968 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43748fd2-22eb-411a-b26b-161e840d414e None admin] PATH_INFO: `/v3/users/b3c975f2cbbb460fad47c5ab1daab49e` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:57.713174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43748fd2-22eb-411a-b26b-161e840d414e None admin] RBAC: Authorizing `identity:get_user(user_id=b3c975f2cbbb460fad47c5ab1daab49e)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:57.714612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43748fd2-22eb-411a-b26b-161e840d414e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:57.716247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 552/1103] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:41:57 2022] GET /identity/v3/users/b3c975f2cbbb460fad47c5ab1daab49e => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:41:57.722188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f1e60-b9ad-4626-9f6c-8ac56caf33a1 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:57.823454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f1e60-b9ad-4626-9f6c-8ac56caf33a1 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:57.841191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:57.859758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f82de5-d6ce-41ac-9d0d-5a8d4b7c3a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:57.860454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f82de5-d6ce-41ac-9d0d-5a8d4b7c3a81 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:57.860741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f82de5-d6ce-41ac-9d0d-5a8d4b7c3a81 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:57.861116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f82de5-d6ce-41ac-9d0d-5a8d4b7c3a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:57.930724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f82de5-d6ce-41ac-9d0d-5a8d4b7c3a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:57.932358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f82de5-d6ce-41ac-9d0d-5a8d4b7c3a81 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:58.035063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 552/1104] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:41:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:41:58.041825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43748fd2-22eb-411a-b26b-161e840d414e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:58.145326 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43748fd2-22eb-411a-b26b-161e840d414e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:58.180512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd995b1d-6ff6-46a7-aa22-162143b43e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:58.181415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd995b1d-6ff6-46a7-aa22-162143b43e32 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:58.181649 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd995b1d-6ff6-46a7-aa22-162143b43e32 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:58.181929 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd995b1d-6ff6-46a7-aa22-162143b43e32 None admin] PATH_INFO: `/v3/projects/cca349ccce3744ca801e4749d66ace8c` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:58.183393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd995b1d-6ff6-46a7-aa22-162143b43e32 None admin] RBAC: Authorizing `identity:get_project(project_id=cca349ccce3744ca801e4749d66ace8c)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:58.184863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd995b1d-6ff6-46a7-aa22-162143b43e32 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:58.186561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 553/1105] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:41:58 2022] GET /identity/v3/projects/cca349ccce3744ca801e4749d66ace8c => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:41:58.193793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f82de5-d6ce-41ac-9d0d-5a8d4b7c3a81 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:41:58.211873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:41:58.293461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f82de5-d6ce-41ac-9d0d-5a8d4b7c3a81 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:41:58.329538 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9fe25-101f-4010-b98f-4499c0972be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:41:58.330191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9fe25-101f-4010-b98f-4499c0972be2 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:41:58.330474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9fe25-101f-4010-b98f-4499c0972be2 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:41:58.330750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9fe25-101f-4010-b98f-4499c0972be2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:41:58.331580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9fe25-101f-4010-b98f-4499c0972be2 None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=cca349ccce3744ca801e4749d66ace8c, user.id=b3c975f2cbbb460fad47c5ab1daab49e) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:41:58.331890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9fe25-101f-4010-b98f-4499c0972be2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:41:58.333346 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9fe25-101f-4010-b98f-4499c0972be2 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:41:58.339534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 553/1106] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:41:58 2022] GET /identity/v3/role_assignments?user.id=b3c975f2cbbb460fad47c5ab1daab49e&scope.project.id=cca349ccce3744ca801e4749d66ace8c&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:00.419464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce6195-2af3-49c9-9e86-8edc02f79e12 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:00.419986 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce6195-2af3-49c9-9e86-8edc02f79e12 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:00.420092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce6195-2af3-49c9-9e86-8edc02f79e12 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:00.421409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 554/1107] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:00.427389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ee719-8e8f-41dd-a4a4-fad82407ddda None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:00.427835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ee719-8e8f-41dd-a4a4-fad82407ddda None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:00.427958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ee719-8e8f-41dd-a4a4-fad82407ddda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:00.445588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-160ee719-8e8f-41dd-a4a4-fad82407ddda None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:00.446815 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-160ee719-8e8f-41dd-a4a4-fad82407ddda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:00.512777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 554/1108] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:00.537735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce6195-2af3-49c9-9e86-8edc02f79e12 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:00.565697 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:00.640068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce6195-2af3-49c9-9e86-8edc02f79e12 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:00.674072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc99c6c-c5c1-4291-90bd-e78a51507b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:00.675451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc99c6c-c5c1-4291-90bd-e78a51507b40 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:00.675736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc99c6c-c5c1-4291-90bd-e78a51507b40 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:00.676011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc99c6c-c5c1-4291-90bd-e78a51507b40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:00.677229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc99c6c-c5c1-4291-90bd-e78a51507b40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:00.678661 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc99c6c-c5c1-4291-90bd-e78a51507b40 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:00.680297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 555/1109] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:42:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:00.686332 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ee719-8e8f-41dd-a4a4-fad82407ddda None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:00.697901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:00.783767 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ee719-8e8f-41dd-a4a4-fad82407ddda None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:00.818260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e846d-2d2e-46df-9a95-0981b5421d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:00.818929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e846d-2d2e-46df-9a95-0981b5421d0c None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:00.819213 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e846d-2d2e-46df-9a95-0981b5421d0c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:00.819490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e846d-2d2e-46df-9a95-0981b5421d0c None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:00.820203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e846d-2d2e-46df-9a95-0981b5421d0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:00.821667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e846d-2d2e-46df-9a95-0981b5421d0c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:00.841517 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 555/1110] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:42:00 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:42:02.917254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509001b-4272-4922-afb8-11714cd50c95 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:02.917813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509001b-4272-4922-afb8-11714cd50c95 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:02.917813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509001b-4272-4922-afb8-11714cd50c95 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:02.919169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 556/1111] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:02.924516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee91c9-3823-4383-949b-c8c7de19d786 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:02.924942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee91c9-3823-4383-949b-c8c7de19d786 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:02.925077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee91c9-3823-4383-949b-c8c7de19d786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:02.942246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-1dee91c9-3823-4383-949b-c8c7de19d786 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:02.943432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1dee91c9-3823-4383-949b-c8c7de19d786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:03.003067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:03.011736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 556/1112] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:03.035798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509001b-4272-4922-afb8-11714cd50c95 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:03.129285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:03.138048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509001b-4272-4922-afb8-11714cd50c95 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:03.172496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8070e-7c10-4b7f-8153-0cdcbcb2fd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:03.173263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8070e-7c10-4b7f-8153-0cdcbcb2fd67 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:03.173541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8070e-7c10-4b7f-8153-0cdcbcb2fd67 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:03.173813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8070e-7c10-4b7f-8153-0cdcbcb2fd67 None admin] PATH_INFO: `/v3/roles/aab847245001401cb4d3d43238298f07` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:03.175285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8070e-7c10-4b7f-8153-0cdcbcb2fd67 None admin] RBAC: Authorizing `identity:get_role(role_id=aab847245001401cb4d3d43238298f07)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:03.176677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8070e-7c10-4b7f-8153-0cdcbcb2fd67 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:03.178041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 557/1113] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:03 2022] GET /identity/v3/roles/aab847245001401cb4d3d43238298f07 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:03.184132 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee91c9-3823-4383-949b-c8c7de19d786 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:03.284833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee91c9-3823-4383-949b-c8c7de19d786 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:03.321355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89b067-d675-4010-84c1-cfacfbcb939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:03.322227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b067-d675-4010-84c1-cfacfbcb939c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:03.322504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b067-d675-4010-84c1-cfacfbcb939c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:03.322776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b067-d675-4010-84c1-cfacfbcb939c None admin] PATH_INFO: `/v3/users/83199c20ba5b4746b9cbcccff02f0b13` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:03.336702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89b067-d675-4010-84c1-cfacfbcb939c None admin] RBAC: Authorizing `identity:get_user(user_id=83199c20ba5b4746b9cbcccff02f0b13)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:03.338162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89b067-d675-4010-84c1-cfacfbcb939c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:03.339881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 557/1114] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:03 2022] GET /identity/v3/users/83199c20ba5b4746b9cbcccff02f0b13 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:03.345871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8070e-7c10-4b7f-8153-0cdcbcb2fd67 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:03.447549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8070e-7c10-4b7f-8153-0cdcbcb2fd67 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:03.482330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2f0c0-0c18-4aae-9bc4-5ed52d8b0a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:03.482977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2f0c0-0c18-4aae-9bc4-5ed52d8b0a32 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:03.483260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2f0c0-0c18-4aae-9bc4-5ed52d8b0a32 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:03.483533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2f0c0-0c18-4aae-9bc4-5ed52d8b0a32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:03.513703 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:03.551876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2f0c0-0c18-4aae-9bc4-5ed52d8b0a32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:03.553350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2f0c0-0c18-4aae-9bc4-5ed52d8b0a32 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:03.655647 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 558/1115] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:42:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:42:03.661637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89b067-d675-4010-84c1-cfacfbcb939c None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:03.716569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:03.760824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89b067-d675-4010-84c1-cfacfbcb939c None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:03.794196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86069cd-0f4b-4535-a1ee-b70bb7f372d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:03.795048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86069cd-0f4b-4535-a1ee-b70bb7f372d6 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:03.795331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86069cd-0f4b-4535-a1ee-b70bb7f372d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:03.795609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86069cd-0f4b-4535-a1ee-b70bb7f372d6 None admin] PATH_INFO: `/v3/projects/cca349ccce3744ca801e4749d66ace8c` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:03.796817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86069cd-0f4b-4535-a1ee-b70bb7f372d6 None admin] RBAC: Authorizing `identity:get_project(project_id=cca349ccce3744ca801e4749d66ace8c)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:03.798238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86069cd-0f4b-4535-a1ee-b70bb7f372d6 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:03.799979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 558/1116] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:42:03 2022] GET /identity/v3/projects/cca349ccce3744ca801e4749d66ace8c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:03.806145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2f0c0-0c18-4aae-9bc4-5ed52d8b0a32 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:03.866671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:03.907095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2f0c0-0c18-4aae-9bc4-5ed52d8b0a32 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:03.942508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869d791-7e25-4da6-89e8-8195c7341344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:03.943173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869d791-7e25-4da6-89e8-8195c7341344 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:03.943434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869d791-7e25-4da6-89e8-8195c7341344 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:03.943712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869d791-7e25-4da6-89e8-8195c7341344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:03.944576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869d791-7e25-4da6-89e8-8195c7341344 None admin] RBAC: Adding query filter params (role.id=aab847245001401cb4d3d43238298f07, scope.project.id=cca349ccce3744ca801e4749d66ace8c, user.id=83199c20ba5b4746b9cbcccff02f0b13) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:03.944891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869d791-7e25-4da6-89e8-8195c7341344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:03.946324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869d791-7e25-4da6-89e8-8195c7341344 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:03.952142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 559/1117] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:42:03 2022] GET /identity/v3/role_assignments?user.id=83199c20ba5b4746b9cbcccff02f0b13&scope.project.id=cca349ccce3744ca801e4749d66ace8c&role.id=aab847245001401cb4d3d43238298f07 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:05.968265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b75b22-c9d5-4b42-803b-d47d11416d92 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:05.968830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b75b22-c9d5-4b42-803b-d47d11416d92 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:05.968830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b75b22-c9d5-4b42-803b-d47d11416d92 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:05.970232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 559/1118] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:05.975892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17ce60-5580-4fc3-a082-26d9f7ead928 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:05.976292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17ce60-5580-4fc3-a082-26d9f7ead928 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:05.976478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17ce60-5580-4fc3-a082-26d9f7ead928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:05.993570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-ac17ce60-5580-4fc3-a082-26d9f7ead928 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:05.994748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ac17ce60-5580-4fc3-a082-26d9f7ead928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:06.063157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 560/1119] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:06.085498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b75b22-c9d5-4b42-803b-d47d11416d92 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:06.187470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b75b22-c9d5-4b42-803b-d47d11416d92 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:06.211294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:06.225042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af584fb-e8cb-42b6-ade6-b16fdf8f6c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:06.225837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af584fb-e8cb-42b6-ade6-b16fdf8f6c7f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:06.226122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af584fb-e8cb-42b6-ade6-b16fdf8f6c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:06.226397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af584fb-e8cb-42b6-ade6-b16fdf8f6c7f None admin] PATH_INFO: `/v3/roles/aab847245001401cb4d3d43238298f07` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:06.227885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af584fb-e8cb-42b6-ade6-b16fdf8f6c7f None admin] RBAC: Authorizing `identity:get_role(role_id=aab847245001401cb4d3d43238298f07)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:06.229349 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af584fb-e8cb-42b6-ade6-b16fdf8f6c7f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:06.230739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 560/1120] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:06 2022] GET /identity/v3/roles/aab847245001401cb4d3d43238298f07 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:06.236721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17ce60-5580-4fc3-a082-26d9f7ead928 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:06.291748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:06.335616 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17ce60-5580-4fc3-a082-26d9f7ead928 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:06.369301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984b50e-728c-4d16-a5bc-9f55d3231235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:06.370178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b50e-728c-4d16-a5bc-9f55d3231235 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:06.370464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b50e-728c-4d16-a5bc-9f55d3231235 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:06.370739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984b50e-728c-4d16-a5bc-9f55d3231235 None admin] PATH_INFO: `/v3/users/83199c20ba5b4746b9cbcccff02f0b13` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:06.371934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984b50e-728c-4d16-a5bc-9f55d3231235 None admin] RBAC: Authorizing `identity:get_user(user_id=83199c20ba5b4746b9cbcccff02f0b13)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:06.373396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984b50e-728c-4d16-a5bc-9f55d3231235 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:06.375034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 561/1121] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:06 2022] GET /identity/v3/users/83199c20ba5b4746b9cbcccff02f0b13 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:06.380864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af584fb-e8cb-42b6-ade6-b16fdf8f6c7f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:06.481088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af584fb-e8cb-42b6-ade6-b16fdf8f6c7f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:06.513563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0e864-ed80-4815-89f3-ee3e18bdc005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:06.514414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e864-ed80-4815-89f3-ee3e18bdc005 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:06.514695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e864-ed80-4815-89f3-ee3e18bdc005 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:06.514967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e864-ed80-4815-89f3-ee3e18bdc005 None admin] PATH_INFO: `/v3/projects/cca349ccce3744ca801e4749d66ace8c` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:06.516122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0e864-ed80-4815-89f3-ee3e18bdc005 None admin] RBAC: Authorizing `identity:get_project(project_id=cca349ccce3744ca801e4749d66ace8c)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:06.517654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0e864-ed80-4815-89f3-ee3e18bdc005 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:06.519958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 561/1122] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:42:06 2022] GET /identity/v3/projects/cca349ccce3744ca801e4749d66ace8c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:06.526353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984b50e-728c-4d16-a5bc-9f55d3231235 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:06.625157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984b50e-728c-4d16-a5bc-9f55d3231235 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:06.649958 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:06.661404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f74da3-a23f-429a-a2ba-fd8f541b1126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:06.662301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f74da3-a23f-429a-a2ba-fd8f541b1126 None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:06.662586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f74da3-a23f-429a-a2ba-fd8f541b1126 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:06.662865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f74da3-a23f-429a-a2ba-fd8f541b1126 None admin] PATH_INFO: `/v3/projects/cca349ccce3744ca801e4749d66ace8c/users/83199c20ba5b4746b9cbcccff02f0b13/roles/aab847245001401cb4d3d43238298f07` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:06.665116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f74da3-a23f-429a-a2ba-fd8f541b1126 None admin] RBAC: Authorizing `identity:create_grant(project_id=cca349ccce3744ca801e4749d66ace8c, user_id=83199c20ba5b4746b9cbcccff02f0b13, role_id=aab847245001401cb4d3d43238298f07)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:06.666547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f74da3-a23f-429a-a2ba-fd8f541b1126 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:06.679263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 562/1123] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:42:06 2022] PUT /identity/v3/projects/cca349ccce3744ca801e4749d66ace8c/users/83199c20ba5b4746b9cbcccff02f0b13/roles/aab847245001401cb4d3d43238298f07 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:42:08.714972 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f7a21-5fb6-49a9-9620-0321f25e9db5 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:08.715549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f7a21-5fb6-49a9-9620-0321f25e9db5 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:08.715549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f7a21-5fb6-49a9-9620-0321f25e9db5 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:08.716934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 562/1124] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:08.722639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c38e0d-6a75-46bb-b21e-a31f7afbdc00 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:08.723060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c38e0d-6a75-46bb-b21e-a31f7afbdc00 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:08.723206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c38e0d-6a75-46bb-b21e-a31f7afbdc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:08.739831 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-77c38e0d-6a75-46bb-b21e-a31f7afbdc00 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:08.741095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-77c38e0d-6a75-46bb-b21e-a31f7afbdc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:08.805292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 563/1125] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:08.829711 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f7a21-5fb6-49a9-9620-0321f25e9db5 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:08.889057 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:08.931452 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f7a21-5fb6-49a9-9620-0321f25e9db5 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:08.965722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a146e03-a76e-45d5-a881-e14a94ae614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:08.966512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a146e03-a76e-45d5-a881-e14a94ae614d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:08.966801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a146e03-a76e-45d5-a881-e14a94ae614d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:08.967080 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a146e03-a76e-45d5-a881-e14a94ae614d None admin] PATH_INFO: `/v3/roles/aab847245001401cb4d3d43238298f07` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:08.968583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a146e03-a76e-45d5-a881-e14a94ae614d None admin] RBAC: Authorizing `identity:get_role(role_id=aab847245001401cb4d3d43238298f07)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:08.969970 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a146e03-a76e-45d5-a881-e14a94ae614d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:08.971380 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 563/1126] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:08 2022] GET /identity/v3/roles/aab847245001401cb4d3d43238298f07 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:08.977196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c38e0d-6a75-46bb-b21e-a31f7afbdc00 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:09.073774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c38e0d-6a75-46bb-b21e-a31f7afbdc00 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:09.088515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:09.109515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e9cf3-c679-47fc-a43c-666edbcb69f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:09.110385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e9cf3-c679-47fc-a43c-666edbcb69f8 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:09.110665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e9cf3-c679-47fc-a43c-666edbcb69f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:09.110937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e9cf3-c679-47fc-a43c-666edbcb69f8 None admin] PATH_INFO: `/v3/users/83199c20ba5b4746b9cbcccff02f0b13` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:09.112708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e9cf3-c679-47fc-a43c-666edbcb69f8 None admin] RBAC: Authorizing `identity:get_user(user_id=83199c20ba5b4746b9cbcccff02f0b13)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:09.114142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e9cf3-c679-47fc-a43c-666edbcb69f8 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:09.115785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 564/1127] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:08 2022] GET /identity/v3/users/83199c20ba5b4746b9cbcccff02f0b13 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:09.121745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a146e03-a76e-45d5-a881-e14a94ae614d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:09.220347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a146e03-a76e-45d5-a881-e14a94ae614d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:09.246026 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:09.254570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceaf8ab-00e3-4115-b80a-64fa2ea91eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:09.255215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceaf8ab-00e3-4115-b80a-64fa2ea91eb1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:09.255496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceaf8ab-00e3-4115-b80a-64fa2ea91eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:09.255768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceaf8ab-00e3-4115-b80a-64fa2ea91eb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:09.320439 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceaf8ab-00e3-4115-b80a-64fa2ea91eb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:09.321878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceaf8ab-00e3-4115-b80a-64fa2ea91eb1 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:09.441408 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 564/1128] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:42:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:42:09.447370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e9cf3-c679-47fc-a43c-666edbcb69f8 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:09.545356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e9cf3-c679-47fc-a43c-666edbcb69f8 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:09.578696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a614668-eb42-4ce8-aac3-4802beb8a9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:09.579615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a614668-eb42-4ce8-aac3-4802beb8a9e3 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:09.579936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a614668-eb42-4ce8-aac3-4802beb8a9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:09.580216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a614668-eb42-4ce8-aac3-4802beb8a9e3 None admin] PATH_INFO: `/v3/projects/cca349ccce3744ca801e4749d66ace8c` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:09.582057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a614668-eb42-4ce8-aac3-4802beb8a9e3 None admin] RBAC: Authorizing `identity:get_project(project_id=cca349ccce3744ca801e4749d66ace8c)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:09.583483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a614668-eb42-4ce8-aac3-4802beb8a9e3 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:09.585208 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 565/1129] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:42:09 2022] GET /identity/v3/projects/cca349ccce3744ca801e4749d66ace8c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:09.591348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceaf8ab-00e3-4115-b80a-64fa2ea91eb1 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:09.617964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:09.690669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceaf8ab-00e3-4115-b80a-64fa2ea91eb1 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:09.725091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e10336-4bea-402a-a247-dd3155acb5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:09.725788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e10336-4bea-402a-a247-dd3155acb5af None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:09.726014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e10336-4bea-402a-a247-dd3155acb5af None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:09.726295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e10336-4bea-402a-a247-dd3155acb5af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:09.727126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e10336-4bea-402a-a247-dd3155acb5af None admin] RBAC: Adding query filter params (role.id=aab847245001401cb4d3d43238298f07, scope.project.id=cca349ccce3744ca801e4749d66ace8c, user.id=83199c20ba5b4746b9cbcccff02f0b13) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:09.727445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e10336-4bea-402a-a247-dd3155acb5af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:09.728906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e10336-4bea-402a-a247-dd3155acb5af None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:09.734680 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 565/1130] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:42:09 2022] GET /identity/v3/role_assignments?user.id=83199c20ba5b4746b9cbcccff02f0b13&scope.project.id=cca349ccce3744ca801e4749d66ace8c&role.id=aab847245001401cb4d3d43238298f07 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:11.888600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3a550-976b-4c9f-a27d-9d0f971936eb None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:11.889168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3a550-976b-4c9f-a27d-9d0f971936eb None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:11.889168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3a550-976b-4c9f-a27d-9d0f971936eb None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:11.890493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 566/1131] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:11.896351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e16ec-4f9e-4c16-be9c-275e9977c26f None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:11.896778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e16ec-4f9e-4c16-be9c-275e9977c26f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:11.896982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e16ec-4f9e-4c16-be9c-275e9977c26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:11.913466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-9a3e16ec-4f9e-4c16-be9c-275e9977c26f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:11.914652 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9a3e16ec-4f9e-4c16-be9c-275e9977c26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:11.979973 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 566/1132] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:12.003701 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3a550-976b-4c9f-a27d-9d0f971936eb None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:12.053409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:12.107534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3a550-976b-4c9f-a27d-9d0f971936eb None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:12.140503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c20e39-9d75-4881-b69a-0aaed4ce0885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:12.141219 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c20e39-9d75-4881-b69a-0aaed4ce0885 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:12.141500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c20e39-9d75-4881-b69a-0aaed4ce0885 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:12.141773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c20e39-9d75-4881-b69a-0aaed4ce0885 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:12.142935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c20e39-9d75-4881-b69a-0aaed4ce0885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:12.145030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c20e39-9d75-4881-b69a-0aaed4ce0885 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:12.146800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 567/1133] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:42:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:12.152685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e16ec-4f9e-4c16-be9c-275e9977c26f None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:12.173971 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:12.253352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e16ec-4f9e-4c16-be9c-275e9977c26f None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:12.285754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce2742-0a56-4069-8d58-34c880af7f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:12.286403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce2742-0a56-4069-8d58-34c880af7f3a None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:12.286686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce2742-0a56-4069-8d58-34c880af7f3a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:12.286962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce2742-0a56-4069-8d58-34c880af7f3a None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:12.287660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce2742-0a56-4069-8d58-34c880af7f3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:12.289191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce2742-0a56-4069-8d58-34c880af7f3a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:12.313870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 567/1134] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:42:12 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:42:14.353742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8bf2c-da97-4b75-8d35-9c24eddafdef None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:14.354262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8bf2c-da97-4b75-8d35-9c24eddafdef None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:14.354354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8bf2c-da97-4b75-8d35-9c24eddafdef None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:14.355631 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 568/1135] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:14.360821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571a9ca-4546-4b94-88f6-6a8a91d49e2c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:14.361231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571a9ca-4546-4b94-88f6-6a8a91d49e2c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:14.361389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571a9ca-4546-4b94-88f6-6a8a91d49e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:14.378345 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-7571a9ca-4546-4b94-88f6-6a8a91d49e2c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:14.379548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7571a9ca-4546-4b94-88f6-6a8a91d49e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:14.447737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 568/1136] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:14.473003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8bf2c-da97-4b75-8d35-9c24eddafdef None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:14.578999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8bf2c-da97-4b75-8d35-9c24eddafdef None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:14.596007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:14.613615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7c446-110b-4f47-9176-810d59c5f9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:14.614403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7c446-110b-4f47-9176-810d59c5f9ef None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:14.614690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7c446-110b-4f47-9176-810d59c5f9ef None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:14.614969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7c446-110b-4f47-9176-810d59c5f9ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:14.616141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7c446-110b-4f47-9176-810d59c5f9ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:14.617721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7c446-110b-4f47-9176-810d59c5f9ef None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:14.619329 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 569/1137] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:42:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:14.625126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571a9ca-4546-4b94-88f6-6a8a91d49e2c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:14.628014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:14.727772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571a9ca-4546-4b94-88f6-6a8a91d49e2c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:14.761448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166037f-411e-424b-8d25-546a3087bd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:14.762114 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166037f-411e-424b-8d25-546a3087bd2a None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:14.762405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166037f-411e-424b-8d25-546a3087bd2a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:14.762686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166037f-411e-424b-8d25-546a3087bd2a None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:14.763404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166037f-411e-424b-8d25-546a3087bd2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:14.764882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166037f-411e-424b-8d25-546a3087bd2a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:14.784912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 569/1138] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:42:14 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:42:16.849072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cab7cf-34d5-4b10-8785-ab9c52d9f1bd None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:16.849614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cab7cf-34d5-4b10-8785-ab9c52d9f1bd None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:16.849614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cab7cf-34d5-4b10-8785-ab9c52d9f1bd None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:16.850971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 570/1139] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:16.856495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf57f9f-676f-4ad0-a9de-d6cfad723b87 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:16.856982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf57f9f-676f-4ad0-a9de-d6cfad723b87 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:16.857074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf57f9f-676f-4ad0-a9de-d6cfad723b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:16.874173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0cf57f9f-676f-4ad0-a9de-d6cfad723b87 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:16.875383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0cf57f9f-676f-4ad0-a9de-d6cfad723b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:16.928623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:16.943007 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 570/1140] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:42:16.966595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cab7cf-34d5-4b10-8785-ab9c52d9f1bd None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:17.069269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cab7cf-34d5-4b10-8785-ab9c52d9f1bd None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:17.102746 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad21282-3197-47ef-9b0b-cf7931234355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:17.103454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad21282-3197-47ef-9b0b-cf7931234355 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:17.103771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad21282-3197-47ef-9b0b-cf7931234355 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:17.104053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad21282-3197-47ef-9b0b-cf7931234355 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:17.114820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad21282-3197-47ef-9b0b-cf7931234355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-4ad21282-3197-47ef-9b0b-cf7931234355 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:42:17.122449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:42:17.123489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:42:17.124379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 571/1141] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:42:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:42:17.129792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf57f9f-676f-4ad0-a9de-d6cfad723b87 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:17.228411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf57f9f-676f-4ad0-a9de-d6cfad723b87 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:17.263254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be061079-f8ff-4cfc-8385-6ff35b08abb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:17.263931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061079-f8ff-4cfc-8385-6ff35b08abb9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:17.264225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061079-f8ff-4cfc-8385-6ff35b08abb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:17.264532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061079-f8ff-4cfc-8385-6ff35b08abb9 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:17.265286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be061079-f8ff-4cfc-8385-6ff35b08abb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:17.265602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be061079-f8ff-4cfc-8385-6ff35b08abb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:17.267003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be061079-f8ff-4cfc-8385-6ff35b08abb9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:17.275625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 571/1142] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:42:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:17.281470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad21282-3197-47ef-9b0b-cf7931234355 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:17.328863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:17.383313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad21282-3197-47ef-9b0b-cf7931234355 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:17.417210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38309eb8-5908-411e-a378-e16660055dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:17.418078 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38309eb8-5908-411e-a378-e16660055dd0 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:17.418356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38309eb8-5908-411e-a378-e16660055dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:17.418625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38309eb8-5908-411e-a378-e16660055dd0 None admin] PATH_INFO: `/v3/users/76798e391aec4f9581417b9c95f69319` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:17.431678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38309eb8-5908-411e-a378-e16660055dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=76798e391aec4f9581417b9c95f69319)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:17.433391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38309eb8-5908-411e-a378-e16660055dd0 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:17.435043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 572/1143] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:17 2022] GET /identity/v3/users/76798e391aec4f9581417b9c95f69319 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:17.441049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be061079-f8ff-4cfc-8385-6ff35b08abb9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:17.491448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:17.542981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be061079-f8ff-4cfc-8385-6ff35b08abb9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:17.576907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb0ddc-b2ef-4db1-94d3-984439ed6135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:17.577562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb0ddc-b2ef-4db1-94d3-984439ed6135 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:17.577852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb0ddc-b2ef-4db1-94d3-984439ed6135 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:17.578132 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb0ddc-b2ef-4db1-94d3-984439ed6135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:17.647316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb0ddc-b2ef-4db1-94d3-984439ed6135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:17.648702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb0ddc-b2ef-4db1-94d3-984439ed6135 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:17.695665 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:17.745206 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 572/1144] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:42:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:42:17.751274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38309eb8-5908-411e-a378-e16660055dd0 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:17.849223 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38309eb8-5908-411e-a378-e16660055dd0 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:17.851573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:17.884438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec028b-9ee5-4737-ac1b-bdcb6ce232ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:17.885293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec028b-9ee5-4737-ac1b-bdcb6ce232ef None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:17.885579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec028b-9ee5-4737-ac1b-bdcb6ce232ef None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:17.885858 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec028b-9ee5-4737-ac1b-bdcb6ce232ef None admin] PATH_INFO: `/v3/projects/dd00251aafe448fe837225602cfb609b` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:17.887043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec028b-9ee5-4737-ac1b-bdcb6ce232ef None admin] RBAC: Authorizing `identity:get_project(project_id=dd00251aafe448fe837225602cfb609b)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:17.888506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec028b-9ee5-4737-ac1b-bdcb6ce232ef None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:17.890197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 573/1145] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:42:17 2022] GET /identity/v3/projects/dd00251aafe448fe837225602cfb609b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:17.896988 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb0ddc-b2ef-4db1-94d3-984439ed6135 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:17.999827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb0ddc-b2ef-4db1-94d3-984439ed6135 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:18.034148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e91b6-1d3c-4eb5-968a-c099b503c394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:18.034797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e91b6-1d3c-4eb5-968a-c099b503c394 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:18.035083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e91b6-1d3c-4eb5-968a-c099b503c394 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:18.035362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e91b6-1d3c-4eb5-968a-c099b503c394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:18.036190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e91b6-1d3c-4eb5-968a-c099b503c394 None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=dd00251aafe448fe837225602cfb609b, user.id=76798e391aec4f9581417b9c95f69319) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:18.036537 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e91b6-1d3c-4eb5-968a-c099b503c394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:18.037957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e91b6-1d3c-4eb5-968a-c099b503c394 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:18.043429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 573/1146] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:42:17 2022] GET /identity/v3/role_assignments?user.id=76798e391aec4f9581417b9c95f69319&scope.project.id=dd00251aafe448fe837225602cfb609b&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:20.048658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401eb31-eea3-43fd-87bb-b714b7bbea82 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:20.049844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401eb31-eea3-43fd-87bb-b714b7bbea82 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:20.049844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401eb31-eea3-43fd-87bb-b714b7bbea82 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:20.051131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 574/1147] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:20.056467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91657c7c-d26a-4ee1-814a-67d34242f462 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:20.056884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91657c7c-d26a-4ee1-814a-67d34242f462 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:20.057039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91657c7c-d26a-4ee1-814a-67d34242f462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:20.073789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-91657c7c-d26a-4ee1-814a-67d34242f462 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:20.074983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-91657c7c-d26a-4ee1-814a-67d34242f462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:20.084004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:20.141760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 574/1148] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:20.164887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401eb31-eea3-43fd-87bb-b714b7bbea82 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:20.265241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401eb31-eea3-43fd-87bb-b714b7bbea82 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:20.298202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f272917-88c4-4995-b293-0260cd04f09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:20.299030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f272917-88c4-4995-b293-0260cd04f09e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:20.299262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f272917-88c4-4995-b293-0260cd04f09e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:20.299583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f272917-88c4-4995-b293-0260cd04f09e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:20.310449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f272917-88c4-4995-b293-0260cd04f09e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-4f272917-88c4-4995-b293-0260cd04f09e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:42:20.318142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:42:20.319181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:42:20.320365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 575/1149] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:42:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:42:20.325622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91657c7c-d26a-4ee1-814a-67d34242f462 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:20.421608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91657c7c-d26a-4ee1-814a-67d34242f462 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:20.454366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917fc21-a4b7-47db-8760-7854fddd0be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:20.455025 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917fc21-a4b7-47db-8760-7854fddd0be8 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:20.455835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917fc21-a4b7-47db-8760-7854fddd0be8 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:20.456113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917fc21-a4b7-47db-8760-7854fddd0be8 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:20.456881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917fc21-a4b7-47db-8760-7854fddd0be8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:20.457185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917fc21-a4b7-47db-8760-7854fddd0be8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:20.458529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917fc21-a4b7-47db-8760-7854fddd0be8 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:20.466393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 575/1150] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:42:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:20.472892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f272917-88c4-4995-b293-0260cd04f09e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:20.503511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:20.572382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f272917-88c4-4995-b293-0260cd04f09e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:20.607171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52ed18-cdd2-4c8c-8b58-b563e5cf922f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:20.608046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52ed18-cdd2-4c8c-8b58-b563e5cf922f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:20.608355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52ed18-cdd2-4c8c-8b58-b563e5cf922f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:20.608647 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52ed18-cdd2-4c8c-8b58-b563e5cf922f None admin] PATH_INFO: `/v3/users/76798e391aec4f9581417b9c95f69319` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:20.609843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52ed18-cdd2-4c8c-8b58-b563e5cf922f None admin] RBAC: Authorizing `identity:get_user(user_id=76798e391aec4f9581417b9c95f69319)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:20.611282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52ed18-cdd2-4c8c-8b58-b563e5cf922f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:20.612974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 576/1151] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:20 2022] GET /identity/v3/users/76798e391aec4f9581417b9c95f69319 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:20.619081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917fc21-a4b7-47db-8760-7854fddd0be8 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:20.628423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:20.723517 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917fc21-a4b7-47db-8760-7854fddd0be8 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:20.756627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17093113-cffd-4594-921f-eb3ff91628c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:20.757482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17093113-cffd-4594-921f-eb3ff91628c9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:20.757769 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17093113-cffd-4594-921f-eb3ff91628c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:20.758047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17093113-cffd-4594-921f-eb3ff91628c9 None admin] PATH_INFO: `/v3/projects/dd00251aafe448fe837225602cfb609b` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:20.759227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17093113-cffd-4594-921f-eb3ff91628c9 None admin] RBAC: Authorizing `identity:get_project(project_id=dd00251aafe448fe837225602cfb609b)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:20.760686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17093113-cffd-4594-921f-eb3ff91628c9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:20.762388 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 576/1152] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:42:20 2022] GET /identity/v3/projects/dd00251aafe448fe837225602cfb609b => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:20.769079 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52ed18-cdd2-4c8c-8b58-b563e5cf922f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:20.866519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:20.869519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52ed18-cdd2-4c8c-8b58-b563e5cf922f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:20.903830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c2dc0-9745-45b7-a41f-919a3ceaf01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:20.904743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c2dc0-9745-45b7-a41f-919a3ceaf01d None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:20.905031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c2dc0-9745-45b7-a41f-919a3ceaf01d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:20.905310 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c2dc0-9745-45b7-a41f-919a3ceaf01d None admin] PATH_INFO: `/v3/projects/dd00251aafe448fe837225602cfb609b/users/76798e391aec4f9581417b9c95f69319/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:20.907352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c2dc0-9745-45b7-a41f-919a3ceaf01d None admin] RBAC: Authorizing `identity:create_grant(project_id=dd00251aafe448fe837225602cfb609b, user_id=76798e391aec4f9581417b9c95f69319, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:20.908880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c2dc0-9745-45b7-a41f-919a3ceaf01d None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:20.922890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 577/1153] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:42:20 2022] PUT /identity/v3/projects/dd00251aafe448fe837225602cfb609b/users/76798e391aec4f9581417b9c95f69319/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:42:22.985261 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08e841-4468-4858-b0d5-9af8980d7363 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:22.985819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08e841-4468-4858-b0d5-9af8980d7363 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:22.985819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08e841-4468-4858-b0d5-9af8980d7363 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:22.987198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 577/1154] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:22.992518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d939d-eb34-4304-adf8-21423b570467 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:22.992972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d939d-eb34-4304-adf8-21423b570467 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:22.993073 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d939d-eb34-4304-adf8-21423b570467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:23.010043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-b09d939d-eb34-4304-adf8-21423b570467 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:23.011323 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b09d939d-eb34-4304-adf8-21423b570467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:23.076562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 578/1155] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:23.100496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08e841-4468-4858-b0d5-9af8980d7363 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:23.175635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:23.201628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08e841-4468-4858-b0d5-9af8980d7363 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:23.235943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6bedd-8d75-4dbd-9e63-679f814fac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:23.236729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6bedd-8d75-4dbd-9e63-679f814fac2f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:23.237018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6bedd-8d75-4dbd-9e63-679f814fac2f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:23.237300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6bedd-8d75-4dbd-9e63-679f814fac2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:23.247182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6bedd-8d75-4dbd-9e63-679f814fac2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-a7a6bedd-8d75-4dbd-9e63-679f814fac2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:42:23.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:42:23.256476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:42:23.257674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 578/1156] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:42:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:42:23.262618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d939d-eb34-4304-adf8-21423b570467 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:23.351861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:23.362077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d939d-eb34-4304-adf8-21423b570467 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:23.395259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883008ef-c8c0-45b8-9607-c3a54d38f33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:23.395920 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883008ef-c8c0-45b8-9607-c3a54d38f33a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:23.396200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883008ef-c8c0-45b8-9607-c3a54d38f33a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:23.396509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883008ef-c8c0-45b8-9607-c3a54d38f33a None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:23.397272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883008ef-c8c0-45b8-9607-c3a54d38f33a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:23.397547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883008ef-c8c0-45b8-9607-c3a54d38f33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:23.398902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883008ef-c8c0-45b8-9607-c3a54d38f33a None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:23.406850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 579/1157] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:42:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:23.412753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6bedd-8d75-4dbd-9e63-679f814fac2f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:23.511010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6bedd-8d75-4dbd-9e63-679f814fac2f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:23.543564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c237f14-5070-495c-acd5-1321d830fb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:23.544451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c237f14-5070-495c-acd5-1321d830fb28 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:23.544738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c237f14-5070-495c-acd5-1321d830fb28 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:23.545016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c237f14-5070-495c-acd5-1321d830fb28 None admin] PATH_INFO: `/v3/users/76798e391aec4f9581417b9c95f69319` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:23.546204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c237f14-5070-495c-acd5-1321d830fb28 None admin] RBAC: Authorizing `identity:get_user(user_id=76798e391aec4f9581417b9c95f69319)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:23.547643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c237f14-5070-495c-acd5-1321d830fb28 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:23.549854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 579/1158] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:23 2022] GET /identity/v3/users/76798e391aec4f9581417b9c95f69319 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:23.556182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883008ef-c8c0-45b8-9607-c3a54d38f33a None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:23.658071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883008ef-c8c0-45b8-9607-c3a54d38f33a None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:23.691811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e812c6-25c4-4357-b875-fa74b380513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:23.692487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e812c6-25c4-4357-b875-fa74b380513e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:23.692775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e812c6-25c4-4357-b875-fa74b380513e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:23.693053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e812c6-25c4-4357-b875-fa74b380513e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:23.713818 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:23.759303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e812c6-25c4-4357-b875-fa74b380513e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:23.760786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e812c6-25c4-4357-b875-fa74b380513e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:23.878682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 580/1159] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:42:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:42:23.884609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c237f14-5070-495c-acd5-1321d830fb28 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:23.887901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:23.983990 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c237f14-5070-495c-acd5-1321d830fb28 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:24.019366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc10dd-16d1-4657-ad28-29073e63e6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:24.020222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc10dd-16d1-4657-ad28-29073e63e6f7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:24.020537 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc10dd-16d1-4657-ad28-29073e63e6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:24.020826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc10dd-16d1-4657-ad28-29073e63e6f7 None admin] PATH_INFO: `/v3/projects/dd00251aafe448fe837225602cfb609b` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:24.022004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc10dd-16d1-4657-ad28-29073e63e6f7 None admin] RBAC: Authorizing `identity:get_project(project_id=dd00251aafe448fe837225602cfb609b)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:24.023433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc10dd-16d1-4657-ad28-29073e63e6f7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:24.025182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 580/1160] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:42:23 2022] GET /identity/v3/projects/dd00251aafe448fe837225602cfb609b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:24.031307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e812c6-25c4-4357-b875-fa74b380513e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:24.083108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:24.129183 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e812c6-25c4-4357-b875-fa74b380513e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:24.165001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ac65e-0e4b-4fc2-991c-f25a91671d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:24.165650 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ac65e-0e4b-4fc2-991c-f25a91671d54 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:24.165930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ac65e-0e4b-4fc2-991c-f25a91671d54 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:24.166204 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ac65e-0e4b-4fc2-991c-f25a91671d54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:24.167029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ac65e-0e4b-4fc2-991c-f25a91671d54 None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=dd00251aafe448fe837225602cfb609b, user.id=76798e391aec4f9581417b9c95f69319) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:24.167343 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ac65e-0e4b-4fc2-991c-f25a91671d54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:24.168816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ac65e-0e4b-4fc2-991c-f25a91671d54 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:24.176030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 581/1161] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:42:24 2022] GET /identity/v3/role_assignments?user.id=76798e391aec4f9581417b9c95f69319&scope.project.id=dd00251aafe448fe837225602cfb609b&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:26.243469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c33f3a-bf37-4049-8369-3ec96267d923 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:26.244109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c33f3a-bf37-4049-8369-3ec96267d923 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:26.244109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c33f3a-bf37-4049-8369-3ec96267d923 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:26.245452 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 581/1162] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:42:26.250992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270758-56d4-459c-83f5-ec001b2ef7d1 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:26.251434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270758-56d4-459c-83f5-ec001b2ef7d1 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:26.251540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270758-56d4-459c-83f5-ec001b2ef7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:26.274620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-52270758-56d4-459c-83f5-ec001b2ef7d1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:26.275847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-52270758-56d4-459c-83f5-ec001b2ef7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:26.343654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 582/1163] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:26.367708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c33f3a-bf37-4049-8369-3ec96267d923 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:26.437928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:26.470230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c33f3a-bf37-4049-8369-3ec96267d923 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:26.504286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0074a-66fb-40ca-bf18-8b0069173e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:26.504964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0074a-66fb-40ca-bf18-8b0069173e63 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:26.505251 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0074a-66fb-40ca-bf18-8b0069173e63 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:26.505528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0074a-66fb-40ca-bf18-8b0069173e63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:26.571689 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0074a-66fb-40ca-bf18-8b0069173e63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:26.573151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0074a-66fb-40ca-bf18-8b0069173e63 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:26.663873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:26.696518 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 582/1164] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:42:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:42:26.702500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52270758-56d4-459c-83f5-ec001b2ef7d1 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:26.749083 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:26.802832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52270758-56d4-459c-83f5-ec001b2ef7d1 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:26.836990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888f6fa-91cb-48d5-adcb-89c13c7561c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:26.837791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888f6fa-91cb-48d5-adcb-89c13c7561c2 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:26.838076 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888f6fa-91cb-48d5-adcb-89c13c7561c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:26.838354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888f6fa-91cb-48d5-adcb-89c13c7561c2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:26.845019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888f6fa-91cb-48d5-adcb-89c13c7561c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:26.846489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888f6fa-91cb-48d5-adcb-89c13c7561c2 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-f888f6fa-91cb-48d5-adcb-89c13c7561c2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:42:26.860537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:42:26.861809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:42:26.862676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 583/1165] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jul 19 20:42:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:42:26.867934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0074a-66fb-40ca-bf18-8b0069173e63 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:26.967203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0074a-66fb-40ca-bf18-8b0069173e63 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:27.000390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7541fc3-ea69-4d54-b21a-405707332c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:27.000980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541fc3-ea69-4d54-b21a-405707332c5b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:27.001267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541fc3-ea69-4d54-b21a-405707332c5b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:27.001548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541fc3-ea69-4d54-b21a-405707332c5b None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:27.002802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7541fc3-ea69-4d54-b21a-405707332c5b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:27.003115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7541fc3-ea69-4d54-b21a-405707332c5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:27.004523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7541fc3-ea69-4d54-b21a-405707332c5b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:27.011812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 583/1166] 10.222.0.85 () {56 vars in 1115 bytes} [Tue Jul 19 20:42:26 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:29.041840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462bd31-cad6-4a67-9e81-b3561ee78230 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:29.042406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462bd31-cad6-4a67-9e81-b3561ee78230 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:29.042406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462bd31-cad6-4a67-9e81-b3561ee78230 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:29.043758 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 584/1167] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:29.049045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b504c-f64b-4091-9038-2e94a2efc6f1 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:29.049491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b504c-f64b-4091-9038-2e94a2efc6f1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:29.049600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b504c-f64b-4091-9038-2e94a2efc6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:29.067950 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-156b504c-f64b-4091-9038-2e94a2efc6f1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:29.069175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-156b504c-f64b-4091-9038-2e94a2efc6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:29.095354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:29.134200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 584/1168] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:29.157585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462bd31-cad6-4a67-9e81-b3561ee78230 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:29.260651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462bd31-cad6-4a67-9e81-b3561ee78230 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:29.268393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:29.297589 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed4085-bbf9-4523-83e1-75943dbc2184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:29.298180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed4085-bbf9-4523-83e1-75943dbc2184 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:29.298465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed4085-bbf9-4523-83e1-75943dbc2184 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:29.298743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed4085-bbf9-4523-83e1-75943dbc2184 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:29.299601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed4085-bbf9-4523-83e1-75943dbc2184 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:29.301000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed4085-bbf9-4523-83e1-75943dbc2184 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:29.315703 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 585/1169] 10.222.0.85 () {60 vars in 1138 bytes} [Tue Jul 19 20:42:29 2022] POST /identity/v3/domains => generated 267 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:42:31.380639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59884c-de54-4e3d-a1bb-1cd05fca876f None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:31.381203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59884c-de54-4e3d-a1bb-1cd05fca876f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:31.381203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59884c-de54-4e3d-a1bb-1cd05fca876f None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:31.382677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 585/1170] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:31.388237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01b2d3-da95-4281-977a-26f99b7f0c16 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:31.388708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01b2d3-da95-4281-977a-26f99b7f0c16 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:31.388820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01b2d3-da95-4281-977a-26f99b7f0c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:31.405488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3c01b2d3-da95-4281-977a-26f99b7f0c16 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:31.406686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3c01b2d3-da95-4281-977a-26f99b7f0c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:31.476350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 586/1171] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:31.500216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59884c-de54-4e3d-a1bb-1cd05fca876f None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:31.604440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59884c-de54-4e3d-a1bb-1cd05fca876f None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:31.639680 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba658ec9-00ed-4faa-8917-0f82ca361929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:31.640726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba658ec9-00ed-4faa-8917-0f82ca361929 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:31.641038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba658ec9-00ed-4faa-8917-0f82ca361929 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:31.641316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba658ec9-00ed-4faa-8917-0f82ca361929 None admin] PATH_INFO: `/v3/domains/239c197159f3490bb6fc6a2ecba7d863` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:31.654424 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba658ec9-00ed-4faa-8917-0f82ca361929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=239c197159f3490bb6fc6a2ecba7d863)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:31.655869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba658ec9-00ed-4faa-8917-0f82ca361929 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:31.657630 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 586/1172] 10.222.0.85 () {56 vars in 1183 bytes} [Tue Jul 19 20:42:31 2022] GET /identity/v3/domains/239c197159f3490bb6fc6a2ecba7d863 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:31.664535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01b2d3-da95-4281-977a-26f99b7f0c16 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:31.759134 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:31.765536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01b2d3-da95-4281-977a-26f99b7f0c16 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:31.800762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421f41e-388e-4551-bb73-b1ab518c8903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:31.801423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421f41e-388e-4551-bb73-b1ab518c8903 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:31.801703 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421f41e-388e-4551-bb73-b1ab518c8903 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:31.801975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421f41e-388e-4551-bb73-b1ab518c8903 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:31.802676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421f41e-388e-4551-bb73-b1ab518c8903 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:31.804095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421f41e-388e-4551-bb73-b1ab518c8903 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:31.830577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 587/1173] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:42:31 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:42:33.915242 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143526a-86f0-4e7a-b9ea-27dd649d8bd3 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:33.915811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143526a-86f0-4e7a-b9ea-27dd649d8bd3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:33.915811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143526a-86f0-4e7a-b9ea-27dd649d8bd3 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:33.917214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 587/1174] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:33.922511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65766b51-46db-4fa6-9c46-8b89c240b5ce None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:33.922944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65766b51-46db-4fa6-9c46-8b89c240b5ce None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:33.923076 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65766b51-46db-4fa6-9c46-8b89c240b5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:33.941336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-65766b51-46db-4fa6-9c46-8b89c240b5ce None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:33.942534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-65766b51-46db-4fa6-9c46-8b89c240b5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:34.009792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 588/1175] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:34.034193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143526a-86f0-4e7a-b9ea-27dd649d8bd3 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:34.061239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:34.138695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143526a-86f0-4e7a-b9ea-27dd649d8bd3 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:34.172686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b455f5a-1149-4d90-b21d-56f38eee7265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:34.173543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b455f5a-1149-4d90-b21d-56f38eee7265 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:34.174331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b455f5a-1149-4d90-b21d-56f38eee7265 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:34.174606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b455f5a-1149-4d90-b21d-56f38eee7265 None admin] PATH_INFO: `/v3/domains/239c197159f3490bb6fc6a2ecba7d863` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:34.175786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b455f5a-1149-4d90-b21d-56f38eee7265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=239c197159f3490bb6fc6a2ecba7d863)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:34.177231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b455f5a-1149-4d90-b21d-56f38eee7265 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:34.178848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 588/1176] 10.222.0.85 () {56 vars in 1183 bytes} [Tue Jul 19 20:42:34 2022] GET /identity/v3/domains/239c197159f3490bb6fc6a2ecba7d863 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:34.184929 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65766b51-46db-4fa6-9c46-8b89c240b5ce None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:34.266368 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:34.289493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65766b51-46db-4fa6-9c46-8b89c240b5ce None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:34.323716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce3c6a-380c-4fdc-8944-f0a9a3863675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:34.324431 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce3c6a-380c-4fdc-8944-f0a9a3863675 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:34.324720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce3c6a-380c-4fdc-8944-f0a9a3863675 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:34.325000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce3c6a-380c-4fdc-8944-f0a9a3863675 None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:34.325712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce3c6a-380c-4fdc-8944-f0a9a3863675 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:34.327154 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce3c6a-380c-4fdc-8944-f0a9a3863675 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:34.347416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 589/1177] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:42:34 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:42:36.428440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03442a-d942-4dc6-a5f7-d1443790b687 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:36.428995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03442a-d942-4dc6-a5f7-d1443790b687 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:36.428995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03442a-d942-4dc6-a5f7-d1443790b687 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:36.430385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 589/1178] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:36.436121 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108b5d-4e71-43e8-8114-eeeb9160fe1a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:36.436636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108b5d-4e71-43e8-8114-eeeb9160fe1a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:36.436741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108b5d-4e71-43e8-8114-eeeb9160fe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:36.453520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-ef108b5d-4e71-43e8-8114-eeeb9160fe1a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:36.454722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ef108b5d-4e71-43e8-8114-eeeb9160fe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:36.521248 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 590/1179] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:36.544642 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03442a-d942-4dc6-a5f7-d1443790b687 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:36.639877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:36.651043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03442a-d942-4dc6-a5f7-d1443790b687 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:36.684495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2cc0c-5319-4daa-a5f4-7314d50237e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:36.685281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2cc0c-5319-4daa-a5f4-7314d50237e0 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:36.685570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2cc0c-5319-4daa-a5f4-7314d50237e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:36.685851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2cc0c-5319-4daa-a5f4-7314d50237e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:36.696148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2cc0c-5319-4daa-a5f4-7314d50237e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-fdd2cc0c-5319-4daa-a5f4-7314d50237e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:42:36.703728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:42:36.704872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:42:36.706879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 590/1180] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:42:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:42:36.711980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef108b5d-4e71-43e8-8114-eeeb9160fe1a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:36.774264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:36.812095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef108b5d-4e71-43e8-8114-eeeb9160fe1a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:36.847285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66338-d18f-490c-87fa-d38d634fc2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:36.847958 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66338-d18f-490c-87fa-d38d634fc2ab None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:36.848242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66338-d18f-490c-87fa-d38d634fc2ab None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:36.848547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66338-d18f-490c-87fa-d38d634fc2ab None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:36.849286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f66338-d18f-490c-87fa-d38d634fc2ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:36.849596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f66338-d18f-490c-87fa-d38d634fc2ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:36.850966 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f66338-d18f-490c-87fa-d38d634fc2ab None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:36.859394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 591/1181] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:42:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:36.865241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2cc0c-5319-4daa-a5f4-7314d50237e0 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:36.964941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2cc0c-5319-4daa-a5f4-7314d50237e0 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:36.998125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430f438-1787-4d05-a973-c7dbab943f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:36.998990 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430f438-1787-4d05-a973-c7dbab943f9f None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:36.999274 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430f438-1787-4d05-a973-c7dbab943f9f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:36.999549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430f438-1787-4d05-a973-c7dbab943f9f None admin] PATH_INFO: `/v3/users/64e917e0f5e146c6b15c452ba1a396de` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:37.013090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430f438-1787-4d05-a973-c7dbab943f9f None admin] RBAC: Authorizing `identity:get_user(user_id=64e917e0f5e146c6b15c452ba1a396de)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:37.014550 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430f438-1787-4d05-a973-c7dbab943f9f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:37.016213 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 591/1182] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:36 2022] GET /identity/v3/users/64e917e0f5e146c6b15c452ba1a396de => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:37.022152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66338-d18f-490c-87fa-d38d634fc2ab None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:37.119289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66338-d18f-490c-87fa-d38d634fc2ab None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:37.144515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:37.153565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f692e4-a1b3-4f45-b125-a95e2945ceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:37.154216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f692e4-a1b3-4f45-b125-a95e2945ceeb None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:37.154502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f692e4-a1b3-4f45-b125-a95e2945ceeb None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:37.154783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f692e4-a1b3-4f45-b125-a95e2945ceeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:37.222890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f692e4-a1b3-4f45-b125-a95e2945ceeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:37.224364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f692e4-a1b3-4f45-b125-a95e2945ceeb None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:37.320516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 592/1183] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:42:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:42:37.326532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430f438-1787-4d05-a973-c7dbab943f9f None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:37.339169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:37.426214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430f438-1787-4d05-a973-c7dbab943f9f None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:37.459906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a79424-d524-45b0-ab11-1e69dd59558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:37.460797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a79424-d524-45b0-ab11-1e69dd59558c None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:37.461087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a79424-d524-45b0-ab11-1e69dd59558c None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:37.461360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a79424-d524-45b0-ab11-1e69dd59558c None admin] PATH_INFO: `/v3/projects/670d916da2324df9aa0b1188a02b1d35` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:37.462553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a79424-d524-45b0-ab11-1e69dd59558c None admin] RBAC: Authorizing `identity:get_project(project_id=670d916da2324df9aa0b1188a02b1d35)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:37.463978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a79424-d524-45b0-ab11-1e69dd59558c None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:37.465717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 592/1184] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:42:37 2022] GET /identity/v3/projects/670d916da2324df9aa0b1188a02b1d35 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:37.471951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f692e4-a1b3-4f45-b125-a95e2945ceeb None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:37.499085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:37.572256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f692e4-a1b3-4f45-b125-a95e2945ceeb None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:37.605253 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c17eda-57ac-484c-a97e-6989a8141c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:37.605894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c17eda-57ac-484c-a97e-6989a8141c81 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:37.606176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c17eda-57ac-484c-a97e-6989a8141c81 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:37.606452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c17eda-57ac-484c-a97e-6989a8141c81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:37.607266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c17eda-57ac-484c-a97e-6989a8141c81 None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=670d916da2324df9aa0b1188a02b1d35, user.id=64e917e0f5e146c6b15c452ba1a396de) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:37.607578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c17eda-57ac-484c-a97e-6989a8141c81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:37.609030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c17eda-57ac-484c-a97e-6989a8141c81 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:37.614341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 593/1185] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:42:37 2022] GET /identity/v3/role_assignments?user.id=64e917e0f5e146c6b15c452ba1a396de&scope.project.id=670d916da2324df9aa0b1188a02b1d35&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:39.622693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a1b6-e0aa-4299-80c4-0a369eeaa30a None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:39.623321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a1b6-e0aa-4299-80c4-0a369eeaa30a None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:39.623321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453a1b6-e0aa-4299-80c4-0a369eeaa30a None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:39.624571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 593/1186] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:39.630205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95951126-fd95-4ecf-923d-a7e1f1d2eb83 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:39.630660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95951126-fd95-4ecf-923d-a7e1f1d2eb83 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:39.630939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95951126-fd95-4ecf-923d-a7e1f1d2eb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:39.650648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-95951126-fd95-4ecf-923d-a7e1f1d2eb83 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:39.651880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-95951126-fd95-4ecf-923d-a7e1f1d2eb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:39.719475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 594/1187] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:42:39.744705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453a1b6-e0aa-4299-80c4-0a369eeaa30a None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:39.841413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:39.864591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453a1b6-e0aa-4299-80c4-0a369eeaa30a None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:39.897708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf7577-b6df-48d6-972a-3c2249165a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:39.898482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf7577-b6df-48d6-972a-3c2249165a63 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:39.898822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf7577-b6df-48d6-972a-3c2249165a63 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:39.899144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf7577-b6df-48d6-972a-3c2249165a63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:39.910046 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf7577-b6df-48d6-972a-3c2249165a63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-4baf7577-b6df-48d6-972a-3c2249165a63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:42:39.917637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:42:39.918747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:42:39.919816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 594/1188] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:42:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:42:39.925172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95951126-fd95-4ecf-923d-a7e1f1d2eb83 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:39.946290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:40.025277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95951126-fd95-4ecf-923d-a7e1f1d2eb83 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:40.059988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee417c0c-6cae-46b9-9cf8-ed1da74e5794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:40.060681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee417c0c-6cae-46b9-9cf8-ed1da74e5794 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:40.060966 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee417c0c-6cae-46b9-9cf8-ed1da74e5794 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:40.061243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee417c0c-6cae-46b9-9cf8-ed1da74e5794 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:40.061985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee417c0c-6cae-46b9-9cf8-ed1da74e5794 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:40.062294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee417c0c-6cae-46b9-9cf8-ed1da74e5794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:40.063665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee417c0c-6cae-46b9-9cf8-ed1da74e5794 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:40.073042 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 595/1189] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:42:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:40.079048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf7577-b6df-48d6-972a-3c2249165a63 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:40.181754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf7577-b6df-48d6-972a-3c2249165a63 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:40.217414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c7bbb-03ba-40e1-8c9a-f074d9de7f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:40.218280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c7bbb-03ba-40e1-8c9a-f074d9de7f69 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:40.218567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c7bbb-03ba-40e1-8c9a-f074d9de7f69 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:40.218846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c7bbb-03ba-40e1-8c9a-f074d9de7f69 None admin] PATH_INFO: `/v3/users/64e917e0f5e146c6b15c452ba1a396de` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:40.220024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c7bbb-03ba-40e1-8c9a-f074d9de7f69 None admin] RBAC: Authorizing `identity:get_user(user_id=64e917e0f5e146c6b15c452ba1a396de)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:40.222100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c7bbb-03ba-40e1-8c9a-f074d9de7f69 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:40.223729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 595/1190] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:40 2022] GET /identity/v3/users/64e917e0f5e146c6b15c452ba1a396de => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:40.229666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee417c0c-6cae-46b9-9cf8-ed1da74e5794 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:40.311232 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:40.331010 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee417c0c-6cae-46b9-9cf8-ed1da74e5794 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:40.363734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825ef72-38d2-4e7a-9657-a19896245fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:40.364634 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825ef72-38d2-4e7a-9657-a19896245fb5 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:40.364918 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825ef72-38d2-4e7a-9657-a19896245fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:40.365195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825ef72-38d2-4e7a-9657-a19896245fb5 None admin] PATH_INFO: `/v3/projects/670d916da2324df9aa0b1188a02b1d35` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:40.366372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825ef72-38d2-4e7a-9657-a19896245fb5 None admin] RBAC: Authorizing `identity:get_project(project_id=670d916da2324df9aa0b1188a02b1d35)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:40.367804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825ef72-38d2-4e7a-9657-a19896245fb5 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:40.369517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 596/1191] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:42:40 2022] GET /identity/v3/projects/670d916da2324df9aa0b1188a02b1d35 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:40.375527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c7bbb-03ba-40e1-8c9a-f074d9de7f69 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:40.385339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:40.477397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c7bbb-03ba-40e1-8c9a-f074d9de7f69 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:40.511098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea548d3-7f98-4acd-b606-4b94720b00db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:40.511997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea548d3-7f98-4acd-b606-4b94720b00db None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:40.512282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea548d3-7f98-4acd-b606-4b94720b00db None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:40.512591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea548d3-7f98-4acd-b606-4b94720b00db None admin] PATH_INFO: `/v3/projects/670d916da2324df9aa0b1188a02b1d35/users/64e917e0f5e146c6b15c452ba1a396de/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:40.514583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea548d3-7f98-4acd-b606-4b94720b00db None admin] RBAC: Authorizing `identity:create_grant(project_id=670d916da2324df9aa0b1188a02b1d35, user_id=64e917e0f5e146c6b15c452ba1a396de, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:40.516013 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea548d3-7f98-4acd-b606-4b94720b00db None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:40.530253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 596/1192] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:42:40 2022] PUT /identity/v3/projects/670d916da2324df9aa0b1188a02b1d35/users/64e917e0f5e146c6b15c452ba1a396de/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:42:42.562497 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e3fbe-df1e-436a-b000-4418a61d0cda None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:42.563003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e3fbe-df1e-436a-b000-4418a61d0cda None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:42.563003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e3fbe-df1e-436a-b000-4418a61d0cda None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:42.564919 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 597/1193] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:42:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:42:42.570141 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eae02-259e-484d-82f9-7d7dc2e2ff94 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:42.570547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eae02-259e-484d-82f9-7d7dc2e2ff94 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:42.570691 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005eae02-259e-484d-82f9-7d7dc2e2ff94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:42.587620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-005eae02-259e-484d-82f9-7d7dc2e2ff94 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:42:42.588841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-005eae02-259e-484d-82f9-7d7dc2e2ff94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:42:42.646631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:42.654935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 597/1194] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:42:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:42:42.678700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e3fbe-df1e-436a-b000-4418a61d0cda None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:42.778830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e3fbe-df1e-436a-b000-4418a61d0cda None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:42.812879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2658455-eafc-45fa-82dc-9c7f38a8703b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:42.813594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2658455-eafc-45fa-82dc-9c7f38a8703b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:42.813876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2658455-eafc-45fa-82dc-9c7f38a8703b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:42.814679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2658455-eafc-45fa-82dc-9c7f38a8703b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:42.824609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2658455-eafc-45fa-82dc-9c7f38a8703b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-a2658455-eafc-45fa-82dc-9c7f38a8703b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:42:42.832160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:42:42.833309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:42:42.834582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 598/1195] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:42:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:42:42.839510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005eae02-259e-484d-82f9-7d7dc2e2ff94 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:42.937452 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005eae02-259e-484d-82f9-7d7dc2e2ff94 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:42.969894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335db90-c146-4449-981e-f9cb2e4959cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:42.970620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335db90-c146-4449-981e-f9cb2e4959cb None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:42.970904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335db90-c146-4449-981e-f9cb2e4959cb None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:42.971179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335db90-c146-4449-981e-f9cb2e4959cb None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:42.971912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335db90-c146-4449-981e-f9cb2e4959cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:42.972217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335db90-c146-4449-981e-f9cb2e4959cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:42.973612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335db90-c146-4449-981e-f9cb2e4959cb None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:42.981528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 598/1196] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:42:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:42.987776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2658455-eafc-45fa-82dc-9c7f38a8703b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:43.000283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:43.085336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2658455-eafc-45fa-82dc-9c7f38a8703b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:43.117498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cc0f7-d549-4c47-a2f8-3c6d788a187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:43.118348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cc0f7-d549-4c47-a2f8-3c6d788a187e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:43.118624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cc0f7-d549-4c47-a2f8-3c6d788a187e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:43.118899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cc0f7-d549-4c47-a2f8-3c6d788a187e None admin] PATH_INFO: `/v3/users/64e917e0f5e146c6b15c452ba1a396de` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:43.120060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cc0f7-d549-4c47-a2f8-3c6d788a187e None admin] RBAC: Authorizing `identity:get_user(user_id=64e917e0f5e146c6b15c452ba1a396de)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:43.121509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cc0f7-d549-4c47-a2f8-3c6d788a187e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:43.123107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 599/1197] 10.222.0.85 () {56 vars in 1177 bytes} [Tue Jul 19 20:42:42 2022] GET /identity/v3/users/64e917e0f5e146c6b15c452ba1a396de => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:43.129469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335db90-c146-4449-981e-f9cb2e4959cb None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:43.187272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:43.230211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335db90-c146-4449-981e-f9cb2e4959cb None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:43.268185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d9c33-abe3-4ffb-bf38-4af0c0945cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:43.268858 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d9c33-abe3-4ffb-bf38-4af0c0945cfe None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:43.269143 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d9c33-abe3-4ffb-bf38-4af0c0945cfe None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:43.269416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d9c33-abe3-4ffb-bf38-4af0c0945cfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:43.334175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d9c33-abe3-4ffb-bf38-4af0c0945cfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:43.335668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d9c33-abe3-4ffb-bf38-4af0c0945cfe None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:43.388937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:43.430043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 599/1198] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:42:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:42:43.436207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cc0f7-d549-4c47-a2f8-3c6d788a187e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:43.512646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:43.538040 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cc0f7-d549-4c47-a2f8-3c6d788a187e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:43.571201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f5653-a0f8-4cec-aa46-1e44a672b875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:43.572054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f5653-a0f8-4cec-aa46-1e44a672b875 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:43.572351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f5653-a0f8-4cec-aa46-1e44a672b875 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:43.572641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f5653-a0f8-4cec-aa46-1e44a672b875 None admin] PATH_INFO: `/v3/projects/670d916da2324df9aa0b1188a02b1d35` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:43.573832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f5653-a0f8-4cec-aa46-1e44a672b875 None admin] RBAC: Authorizing `identity:get_project(project_id=670d916da2324df9aa0b1188a02b1d35)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:43.575265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f5653-a0f8-4cec-aa46-1e44a672b875 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:43.576982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 600/1199] 10.222.0.85 () {56 vars in 1186 bytes} [Tue Jul 19 20:42:43 2022] GET /identity/v3/projects/670d916da2324df9aa0b1188a02b1d35 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:43.583158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d9c33-abe3-4ffb-bf38-4af0c0945cfe None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:42:43.682822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d9c33-abe3-4ffb-bf38-4af0c0945cfe None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:42:43.716510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f937894-da88-4e7d-ad7c-20be68e1f78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:42:43.717165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937894-da88-4e7d-ad7c-20be68e1f78e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:42:43.717454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937894-da88-4e7d-ad7c-20be68e1f78e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:42:43.717734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f937894-da88-4e7d-ad7c-20be68e1f78e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:42:43.718567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f937894-da88-4e7d-ad7c-20be68e1f78e None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=670d916da2324df9aa0b1188a02b1d35, user.id=64e917e0f5e146c6b15c452ba1a396de) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:42:43.718878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f937894-da88-4e7d-ad7c-20be68e1f78e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:42:43.720295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f937894-da88-4e7d-ad7c-20be68e1f78e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:42:43.726713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 600/1200] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:42:43 2022] GET /identity/v3/role_assignments?user.id=64e917e0f5e146c6b15c452ba1a396de&scope.project.id=670d916da2324df9aa0b1188a02b1d35&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:42:53.567100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=27 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:42:53.724074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:39.734888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f97234-59cc-41a7-901a-69845be9b7e2 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:39.735482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f97234-59cc-41a7-901a-69845be9b7e2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:39.735482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f97234-59cc-41a7-901a-69845be9b7e2 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:39.737053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 601/1201] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:43:39.743000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa8fcf-52cd-4ade-93d6-07b09f2536e1 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:39.743566 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa8fcf-52cd-4ade-93d6-07b09f2536e1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:39.743566 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa8fcf-52cd-4ade-93d6-07b09f2536e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:39.769148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-dffa8fcf-52cd-4ade-93d6-07b09f2536e1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:39.770600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-dffa8fcf-52cd-4ade-93d6-07b09f2536e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:39.872376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 601/1202] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:43:39.900211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f97234-59cc-41a7-901a-69845be9b7e2 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:40.072153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f97234-59cc-41a7-901a-69845be9b7e2 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:40.137424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec067cd9-7fe8-470a-8dfb-8a8fab09172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:40.139542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec067cd9-7fe8-470a-8dfb-8a8fab09172d None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:40.139953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec067cd9-7fe8-470a-8dfb-8a8fab09172d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:40.140502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec067cd9-7fe8-470a-8dfb-8a8fab09172d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:40.158910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec067cd9-7fe8-470a-8dfb-8a8fab09172d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:40.161269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec067cd9-7fe8-470a-8dfb-8a8fab09172d None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-ec067cd9-7fe8-470a-8dfb-8a8fab09172d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:43:40.184776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:40.186064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:40.193195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:43:40.193195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:43:40.193195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:40.193195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:43:40.193195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 602/1203] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:43:40.193592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa8fcf-52cd-4ade-93d6-07b09f2536e1 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:40.336580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa8fcf-52cd-4ade-93d6-07b09f2536e1 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:40.413687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:40.428397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0eefcc-6ad2-4389-9982-44b6399784fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:40.429364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eefcc-6ad2-4389-9982-44b6399784fc None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:40.429656 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eefcc-6ad2-4389-9982-44b6399784fc None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:40.429936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0eefcc-6ad2-4389-9982-44b6399784fc None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:40.430720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0eefcc-6ad2-4389-9982-44b6399784fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:40.430993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0eefcc-6ad2-4389-9982-44b6399784fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:40.432449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0eefcc-6ad2-4389-9982-44b6399784fc None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:40.453965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 602/1204] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:40.455692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec067cd9-7fe8-470a-8dfb-8a8fab09172d None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:40.555361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:40.647134 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec067cd9-7fe8-470a-8dfb-8a8fab09172d None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:40.720158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcbaeb-d31f-439d-b2fb-149cdadc410e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:40.721190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcbaeb-d31f-439d-b2fb-149cdadc410e None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:40.721551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcbaeb-d31f-439d-b2fb-149cdadc410e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:40.721884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcbaeb-d31f-439d-b2fb-149cdadc410e None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:40.722773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcbaeb-d31f-439d-b2fb-149cdadc410e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:40.724486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcbaeb-d31f-439d-b2fb-149cdadc410e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:40.767514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 603/1205] 10.222.0.85 () {60 vars in 1132 bytes} [Tue Jul 19 20:43:40 2022] POST /identity/v3/users => generated 251 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:43:43.018523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd13997-5f02-447a-9463-824702113362 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:43.019097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd13997-5f02-447a-9463-824702113362 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:43.019097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd13997-5f02-447a-9463-824702113362 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:43.020687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 603/1206] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:43:43.026050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902058b0-28b4-40cb-b0a1-be6d0aa52339 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:43.026560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902058b0-28b4-40cb-b0a1-be6d0aa52339 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:43.026665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902058b0-28b4-40cb-b0a1-be6d0aa52339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:43.046823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-902058b0-28b4-40cb-b0a1-be6d0aa52339 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:43.048031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-902058b0-28b4-40cb-b0a1-be6d0aa52339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:43.130003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 604/1207] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:43:43.155837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd13997-5f02-447a-9463-824702113362 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:43.308457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd13997-5f02-447a-9463-824702113362 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:43.351891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac04f8d-8792-4c32-b366-9b3d3391ad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:43.352757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac04f8d-8792-4c32-b366-9b3d3391ad01 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:43.353155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac04f8d-8792-4c32-b366-9b3d3391ad01 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:43.353266 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac04f8d-8792-4c32-b366-9b3d3391ad01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:43.370857 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac04f8d-8792-4c32-b366-9b3d3391ad01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-6ac04f8d-8792-4c32-b366-9b3d3391ad01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:43:43.385911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:43.387045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:43:43.387941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:43:43.389284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 604/1208] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:43:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:43.396828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902058b0-28b4-40cb-b0a1-be6d0aa52339 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:43.445754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:43.515456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902058b0-28b4-40cb-b0a1-be6d0aa52339 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:43.554456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86383e1b-c9df-498b-a540-48ce32992d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:43.555131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383e1b-c9df-498b-a540-48ce32992d25 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:43.555415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383e1b-c9df-498b-a540-48ce32992d25 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:43.555687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383e1b-c9df-498b-a540-48ce32992d25 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:43.556465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86383e1b-c9df-498b-a540-48ce32992d25 None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:43.556780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86383e1b-c9df-498b-a540-48ce32992d25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:43.558615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86383e1b-c9df-498b-a540-48ce32992d25 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:43.568201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 605/1209] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:43:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:43.573792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac04f8d-8792-4c32-b366-9b3d3391ad01 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:43.641297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:43.705430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac04f8d-8792-4c32-b366-9b3d3391ad01 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:43.741786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac2137-588f-40c4-aa4d-01aa2e96661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:43.742590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac2137-588f-40c4-aa4d-01aa2e96661d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:43.742936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac2137-588f-40c4-aa4d-01aa2e96661d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:43.743205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac2137-588f-40c4-aa4d-01aa2e96661d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:43.754712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac2137-588f-40c4-aa4d-01aa2e96661d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:43.756276 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac2137-588f-40c4-aa4d-01aa2e96661d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-86ac2137-588f-40c4-aa4d-01aa2e96661d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:43:43.775071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:43.776240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:43.778171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:43:43.778171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:43:43.778171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:43.778171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:43:43.778171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 605/1210] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:43:43.783145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86383e1b-c9df-498b-a540-48ce32992d25 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:43.894718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:43.898988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86383e1b-c9df-498b-a540-48ce32992d25 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:43.935488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a77c9-b19b-4206-ab27-f7d551bdd433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:43.937221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a77c9-b19b-4206-ab27-f7d551bdd433 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:43.937221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a77c9-b19b-4206-ab27-f7d551bdd433 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:43.937221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a77c9-b19b-4206-ab27-f7d551bdd433 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:43.937814 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a77c9-b19b-4206-ab27-f7d551bdd433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:43.938128 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a77c9-b19b-4206-ab27-f7d551bdd433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:43.939510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a77c9-b19b-4206-ab27-f7d551bdd433 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:43.951744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 606/1211] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:43:43.958045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac2137-588f-40c4-aa4d-01aa2e96661d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:44.091908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac2137-588f-40c4-aa4d-01aa2e96661d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:44.093765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:44.141785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038103e8-3144-4ef5-9b5a-4d7117849246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:44.142719 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038103e8-3144-4ef5-9b5a-4d7117849246 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:44.143006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038103e8-3144-4ef5-9b5a-4d7117849246 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:44.143342 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038103e8-3144-4ef5-9b5a-4d7117849246 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:44.152727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038103e8-3144-4ef5-9b5a-4d7117849246 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:44.154490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038103e8-3144-4ef5-9b5a-4d7117849246 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-038103e8-3144-4ef5-9b5a-4d7117849246 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:43:44.176755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:44.177894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:44.178868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:44.178868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:44.178868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:43:44.178868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:43:44.178868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:43:44.178868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:43:44.178868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:43:44.178868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:43:44.178868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:43:44.178868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:43:44.179621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 606/1212] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:44.185094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a77c9-b19b-4206-ab27-f7d551bdd433 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:44.305189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a77c9-b19b-4206-ab27-f7d551bdd433 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:44.342370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef1f67-4dcc-415f-b1e7-6ac47823a510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:44.343049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef1f67-4dcc-415f-b1e7-6ac47823a510 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:44.343327 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef1f67-4dcc-415f-b1e7-6ac47823a510 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:44.343657 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef1f67-4dcc-415f-b1e7-6ac47823a510 None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:44.345009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef1f67-4dcc-415f-b1e7-6ac47823a510 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:44.346109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef1f67-4dcc-415f-b1e7-6ac47823a510 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:44.346738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef1f67-4dcc-415f-b1e7-6ac47823a510 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:44.367065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 607/1213] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jul 19 20:43:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:44.373312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038103e8-3144-4ef5-9b5a-4d7117849246 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:44.513925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038103e8-3144-4ef5-9b5a-4d7117849246 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:44.557131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20675db8-d9ce-4dac-9762-12762eb3b169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:44.558065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675db8-d9ce-4dac-9762-12762eb3b169 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:44.558352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675db8-d9ce-4dac-9762-12762eb3b169 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:44.558636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675db8-d9ce-4dac-9762-12762eb3b169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:44.601234 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:44.663205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20675db8-d9ce-4dac-9762-12762eb3b169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:44.664718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20675db8-d9ce-4dac-9762-12762eb3b169 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:44.807307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 607/1214] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:43:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 20:43:44.813592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef1f67-4dcc-415f-b1e7-6ac47823a510 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:44.834709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:44.931142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef1f67-4dcc-415f-b1e7-6ac47823a510 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:44.971694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3790734-5afd-45f0-afd5-5ce516ad8a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:44.972807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3790734-5afd-45f0-afd5-5ce516ad8a6f None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:44.973056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3790734-5afd-45f0-afd5-5ce516ad8a6f None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:44.975018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3790734-5afd-45f0-afd5-5ce516ad8a6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:44.985641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3790734-5afd-45f0-afd5-5ce516ad8a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:44.987307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3790734-5afd-45f0-afd5-5ce516ad8a6f None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-e3790734-5afd-45f0-afd5-5ce516ad8a6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:43:45.009258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:45.010800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:45.012070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:43:45.012070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:43:45.012070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:45.012070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:43:45.015530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 608/1215] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:43:45.017105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20675db8-d9ce-4dac-9762-12762eb3b169 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:45.091762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:45.138554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20675db8-d9ce-4dac-9762-12762eb3b169 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:45.181770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0346c41-fe7f-4ba5-ae6b-fbdb5f6654dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:45.182448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0346c41-fe7f-4ba5-ae6b-fbdb5f6654dc None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:45.183021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0346c41-fe7f-4ba5-ae6b-fbdb5f6654dc None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:45.183326 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0346c41-fe7f-4ba5-ae6b-fbdb5f6654dc None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:45.184018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0346c41-fe7f-4ba5-ae6b-fbdb5f6654dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:45.184335 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0346c41-fe7f-4ba5-ae6b-fbdb5f6654dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:45.185820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0346c41-fe7f-4ba5-ae6b-fbdb5f6654dc None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:45.200157 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 608/1216] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:45.209167 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3790734-5afd-45f0-afd5-5ce516ad8a6f None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:45.287075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:45.320931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3790734-5afd-45f0-afd5-5ce516ad8a6f None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:45.360672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8ba3d-06ab-4c35-a494-7735a72eec33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:45.360672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8ba3d-06ab-4c35-a494-7735a72eec33 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:45.360672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8ba3d-06ab-4c35-a494-7735a72eec33 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:45.361079 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8ba3d-06ab-4c35-a494-7735a72eec33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:45.368787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8ba3d-06ab-4c35-a494-7735a72eec33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:45.370482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8ba3d-06ab-4c35-a494-7735a72eec33 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-b5e8ba3d-06ab-4c35-a494-7735a72eec33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:43:45.385949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:45.386999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:45.387917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:43:45.387917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:43:45.387917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:43:45.387917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:43:45.387917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:43:45.387917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:43:45.387917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:43:45.387917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:43:45.387917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 609/1217] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:43:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:45.395259 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0346c41-fe7f-4ba5-ae6b-fbdb5f6654dc None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:45.520662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0346c41-fe7f-4ba5-ae6b-fbdb5f6654dc None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:45.546808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:45.561041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e6d95-708f-4fd5-8ee0-3feea980d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:45.561712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e6d95-708f-4fd5-8ee0-3feea980d338 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:45.562001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e6d95-708f-4fd5-8ee0-3feea980d338 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:45.562280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e6d95-708f-4fd5-8ee0-3feea980d338 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:45.563125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e6d95-708f-4fd5-8ee0-3feea980d338 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:45.563452 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e6d95-708f-4fd5-8ee0-3feea980d338 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:45.564945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e6d95-708f-4fd5-8ee0-3feea980d338 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:45.579908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 609/1218] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:43:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:45.585837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8ba3d-06ab-4c35-a494-7735a72eec33 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:45.705635 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8ba3d-06ab-4c35-a494-7735a72eec33 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:45.736856 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:45.744375 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a7e65-2944-4b06-88c5-93449883c2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:45.744375 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a7e65-2944-4b06-88c5-93449883c2a4 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:45.744910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a7e65-2944-4b06-88c5-93449883c2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:45.745293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a7e65-2944-4b06-88c5-93449883c2a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:45.746243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a7e65-2944-4b06-88c5-93449883c2a4 None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=fffebfc51dba47f68263c0bcd6b18d22) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:45.746570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a7e65-2944-4b06-88c5-93449883c2a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:45.748024 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a7e65-2944-4b06-88c5-93449883c2a4 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:45.755312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 610/1219] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:43:45 2022] GET /identity/v3/role_assignments?user.id=fffebfc51dba47f68263c0bcd6b18d22&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:43:47.973468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf412a2-f865-4115-a43b-12d83ca0463c None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:47.974040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf412a2-f865-4115-a43b-12d83ca0463c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:47.974040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf412a2-f865-4115-a43b-12d83ca0463c None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:47.975512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 610/1220] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:43:47.981917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747b6cb-00cf-4774-afeb-e681e03a8480 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:47.982580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747b6cb-00cf-4774-afeb-e681e03a8480 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:47.982580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747b6cb-00cf-4774-afeb-e681e03a8480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:48.009054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-7747b6cb-00cf-4774-afeb-e681e03a8480 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:48.010890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7747b6cb-00cf-4774-afeb-e681e03a8480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:48.121681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 611/1221] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:43:48.147333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf412a2-f865-4115-a43b-12d83ca0463c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:48.293882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf412a2-f865-4115-a43b-12d83ca0463c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:48.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3783f2a-6933-4821-b858-3da7e657012e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:48.341011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3783f2a-6933-4821-b858-3da7e657012e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:48.341434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3783f2a-6933-4821-b858-3da7e657012e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:48.341637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3783f2a-6933-4821-b858-3da7e657012e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:48.356436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3783f2a-6933-4821-b858-3da7e657012e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-d3783f2a-6933-4821-b858-3da7e657012e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:43:48.364508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:48.366480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:43:48.368477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 611/1222] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:43:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:48.373805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747b6cb-00cf-4774-afeb-e681e03a8480 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:48.497627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747b6cb-00cf-4774-afeb-e681e03a8480 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:48.533973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:48.548234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553fad1-7eea-4321-98d8-093f3728fba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:48.549065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553fad1-7eea-4321-98d8-093f3728fba6 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:48.549492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553fad1-7eea-4321-98d8-093f3728fba6 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:48.549858 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553fad1-7eea-4321-98d8-093f3728fba6 None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:48.550817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553fad1-7eea-4321-98d8-093f3728fba6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:48.551272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553fad1-7eea-4321-98d8-093f3728fba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:48.552968 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553fad1-7eea-4321-98d8-093f3728fba6 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:48.564680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 612/1223] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:43:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:48.571312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3783f2a-6933-4821-b858-3da7e657012e None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:48.631735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:48.710644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3783f2a-6933-4821-b858-3da7e657012e None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:48.748861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9d201-78bd-4f04-a634-856566699f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:48.749706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9d201-78bd-4f04-a634-856566699f15 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:48.750515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9d201-78bd-4f04-a634-856566699f15 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:48.750797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9d201-78bd-4f04-a634-856566699f15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:48.762216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9d201-78bd-4f04-a634-856566699f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:48.763779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9d201-78bd-4f04-a634-856566699f15 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-fbf9d201-78bd-4f04-a634-856566699f15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:43:48.782908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:48.784175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:48.785127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:43:48.785127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:43:48.785127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:48.785127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:43:48.785127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 612/1224] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:48.790481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553fad1-7eea-4321-98d8-093f3728fba6 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:48.899354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553fad1-7eea-4321-98d8-093f3728fba6 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:48.936836 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f3859-8f7e-411e-ae8a-5701e02fc341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:48.937533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f3859-8f7e-411e-ae8a-5701e02fc341 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:48.937866 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f3859-8f7e-411e-ae8a-5701e02fc341 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:48.938096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f3859-8f7e-411e-ae8a-5701e02fc341 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:48.938811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f3859-8f7e-411e-ae8a-5701e02fc341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:48.939120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f3859-8f7e-411e-ae8a-5701e02fc341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:48.940514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f3859-8f7e-411e-ae8a-5701e02fc341 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:48.954326 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 613/1225] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:48.960132 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9d201-78bd-4f04-a634-856566699f15 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:49.066708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:49.074738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9d201-78bd-4f04-a634-856566699f15 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:49.117300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbfd41-8923-43c0-9249-e59d03f29c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:49.118127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbfd41-8923-43c0-9249-e59d03f29c9a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:49.118423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbfd41-8923-43c0-9249-e59d03f29c9a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:49.118693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbfd41-8923-43c0-9249-e59d03f29c9a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:49.126996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbfd41-8923-43c0-9249-e59d03f29c9a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:49.128516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbfd41-8923-43c0-9249-e59d03f29c9a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-4efbfd41-8923-43c0-9249-e59d03f29c9a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:43:49.146483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:49.147682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:49.148878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:49.148878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:49.148878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:43:49.148878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:43:49.148878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:43:49.148878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:43:49.148878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:43:49.148878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:43:49.148878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:43:49.148878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:43:49.148878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 613/1226] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:49.154063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f3859-8f7e-411e-ae8a-5701e02fc341 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:49.207928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:49.283278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f3859-8f7e-411e-ae8a-5701e02fc341 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:49.330627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf840d-50a5-47cc-b240-0b621e65231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:49.331301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf840d-50a5-47cc-b240-0b621e65231e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:49.331601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf840d-50a5-47cc-b240-0b621e65231e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:49.331879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf840d-50a5-47cc-b240-0b621e65231e None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:49.332736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf840d-50a5-47cc-b240-0b621e65231e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:49.333038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf840d-50a5-47cc-b240-0b621e65231e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:49.334478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf840d-50a5-47cc-b240-0b621e65231e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:49.353844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 614/1227] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jul 19 20:43:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:49.359577 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbfd41-8923-43c0-9249-e59d03f29c9a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:49.472924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbfd41-8923-43c0-9249-e59d03f29c9a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:49.515314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adf3eb-f618-4f5a-a87e-3606a2da87df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:49.516723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adf3eb-f618-4f5a-a87e-3606a2da87df None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:49.517011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adf3eb-f618-4f5a-a87e-3606a2da87df None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:49.517290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adf3eb-f618-4f5a-a87e-3606a2da87df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:49.529680 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adf3eb-f618-4f5a-a87e-3606a2da87df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:49.531435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adf3eb-f618-4f5a-a87e-3606a2da87df None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-52adf3eb-f618-4f5a-a87e-3606a2da87df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:43:49.550850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:49.552023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:49.552999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:43:49.552999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:43:49.552999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:49.552999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:43:49.552999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 614/1228] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:49.558521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf840d-50a5-47cc-b240-0b621e65231e None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:49.675492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf840d-50a5-47cc-b240-0b621e65231e None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:49.678503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:49.716588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e696b5-8643-4112-8f5c-9900a989509d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:49.717333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e696b5-8643-4112-8f5c-9900a989509d None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:49.717642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e696b5-8643-4112-8f5c-9900a989509d None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:49.718013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e696b5-8643-4112-8f5c-9900a989509d None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:49.719411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e696b5-8643-4112-8f5c-9900a989509d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:49.719730 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e696b5-8643-4112-8f5c-9900a989509d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:49.721361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e696b5-8643-4112-8f5c-9900a989509d None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:49.736588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 615/1229] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:49.742299 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adf3eb-f618-4f5a-a87e-3606a2da87df None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:49.752795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:49.866969 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adf3eb-f618-4f5a-a87e-3606a2da87df None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:49.916040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b9f48-2b97-41ed-a254-6a0966219027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:49.916927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b9f48-2b97-41ed-a254-6a0966219027 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:49.917219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b9f48-2b97-41ed-a254-6a0966219027 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:49.917596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b9f48-2b97-41ed-a254-6a0966219027 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:49.925951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b9f48-2b97-41ed-a254-6a0966219027 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:49.927562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b9f48-2b97-41ed-a254-6a0966219027 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-0a9b9f48-2b97-41ed-a254-6a0966219027 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:43:49.943953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:49.945887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:49.947626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:43:49.947626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:43:49.947626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:43:49.947626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:43:49.947626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:43:49.947626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:43:49.947626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:43:49.947626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:43:49.947626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 615/1230] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:43:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:49.951849 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e696b5-8643-4112-8f5c-9900a989509d None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:50.081890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e696b5-8643-4112-8f5c-9900a989509d None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:50.130659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59671523-b945-47e7-ba5f-662c2db16e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:50.131455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59671523-b945-47e7-ba5f-662c2db16e56 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:50.131936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59671523-b945-47e7-ba5f-662c2db16e56 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:50.132218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59671523-b945-47e7-ba5f-662c2db16e56 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:50.133241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59671523-b945-47e7-ba5f-662c2db16e56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:50.133625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59671523-b945-47e7-ba5f-662c2db16e56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:50.135727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59671523-b945-47e7-ba5f-662c2db16e56 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:50.150617 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 616/1231] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:43:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:50.157467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b9f48-2b97-41ed-a254-6a0966219027 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:50.248067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:50.290811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b9f48-2b97-41ed-a254-6a0966219027 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:50.333918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7107850-a2c8-4d98-a18d-ed3499062e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:50.335160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7107850-a2c8-4d98-a18d-ed3499062e1f None admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:50.335634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7107850-a2c8-4d98-a18d-ed3499062e1f None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:50.335965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7107850-a2c8-4d98-a18d-ed3499062e1f None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/fffebfc51dba47f68263c0bcd6b18d22/roles/b2e0a9d2d13844e980a69f441f36d492` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:50.353704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7107850-a2c8-4d98-a18d-ed3499062e1f None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=fffebfc51dba47f68263c0bcd6b18d22, role_id=b2e0a9d2d13844e980a69f441f36d492)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:50.355501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7107850-a2c8-4d98-a18d-ed3499062e1f None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:50.374802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 616/1232] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:43:50 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/fffebfc51dba47f68263c0bcd6b18d22/roles/b2e0a9d2d13844e980a69f441f36d492 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:43:52.522593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06208e3-47ef-40ad-b814-5c94714cb2c5 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:52.523198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06208e3-47ef-40ad-b814-5c94714cb2c5 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:52.523198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06208e3-47ef-40ad-b814-5c94714cb2c5 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:52.524823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 617/1233] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:43:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:43:52.536596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcd2a5-09fb-41f8-b4ca-3b434e401f74 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:52.537183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcd2a5-09fb-41f8-b4ca-3b434e401f74 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:52.537183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcd2a5-09fb-41f8-b4ca-3b434e401f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:52.556935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-01bcd2a5-09fb-41f8-b4ca-3b434e401f74 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:52.558477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-01bcd2a5-09fb-41f8-b4ca-3b434e401f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:52.637999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 617/1234] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:43:52.670519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06208e3-47ef-40ad-b814-5c94714cb2c5 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:52.715225 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:52.816148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06208e3-47ef-40ad-b814-5c94714cb2c5 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:52.876414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc11c5-d44c-4397-bc50-067276e9f27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:52.877523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc11c5-d44c-4397-bc50-067276e9f27a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:52.877910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc11c5-d44c-4397-bc50-067276e9f27a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:52.878277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc11c5-d44c-4397-bc50-067276e9f27a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:52.897147 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc11c5-d44c-4397-bc50-067276e9f27a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-22fc11c5-d44c-4397-bc50-067276e9f27a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:43:52.910029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:52.911221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:43:52.912247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 618/1235] 10.222.0.85 () {56 vars in 1102 bytes} [Tue Jul 19 20:43:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:52.923166 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcd2a5-09fb-41f8-b4ca-3b434e401f74 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:52.976575 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:53.046449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcd2a5-09fb-41f8-b4ca-3b434e401f74 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:53.091012 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6887ef-f1cc-498e-a885-15c235a8362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:53.091761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6887ef-f1cc-498e-a885-15c235a8362d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:53.092394 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6887ef-f1cc-498e-a885-15c235a8362d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:53.092741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6887ef-f1cc-498e-a885-15c235a8362d None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:53.094178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6887ef-f1cc-498e-a885-15c235a8362d None admin] RBAC: Adding query filter params (name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:53.094582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6887ef-f1cc-498e-a885-15c235a8362d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:53.096146 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6887ef-f1cc-498e-a885-15c235a8362d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:53.105825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 618/1236] 10.222.0.85 () {56 vars in 1103 bytes} [Tue Jul 19 20:43:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:53.112544 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc11c5-d44c-4397-bc50-067276e9f27a None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:53.223655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:53.247074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc11c5-d44c-4397-bc50-067276e9f27a None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:53.293027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b86a0-afbf-4c8e-a24c-d1733026987a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:53.294098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b86a0-afbf-4c8e-a24c-d1733026987a None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:53.294519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b86a0-afbf-4c8e-a24c-d1733026987a None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:53.294839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b86a0-afbf-4c8e-a24c-d1733026987a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:53.310370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b86a0-afbf-4c8e-a24c-d1733026987a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:53.312198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b86a0-afbf-4c8e-a24c-d1733026987a None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-019b86a0-afbf-4c8e-a24c-d1733026987a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:43:53.333716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:53.335036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:53.336023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:43:53.336023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:43:53.336023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:53.336023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:43:53.336023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 619/1237] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:43:53.344370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6887ef-f1cc-498e-a885-15c235a8362d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:53.464432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6887ef-f1cc-498e-a885-15c235a8362d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:53.474972 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:53.501908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706926d-e177-493c-a74f-8460a54575cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:53.502641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706926d-e177-493c-a74f-8460a54575cc None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:53.502930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706926d-e177-493c-a74f-8460a54575cc None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:53.503208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706926d-e177-493c-a74f-8460a54575cc None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:53.504072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706926d-e177-493c-a74f-8460a54575cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:53.504430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706926d-e177-493c-a74f-8460a54575cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:53.506033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706926d-e177-493c-a74f-8460a54575cc None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:53.527199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 619/1238] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:53.528474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b86a0-afbf-4c8e-a24c-d1733026987a None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:53.645846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b86a0-afbf-4c8e-a24c-d1733026987a None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:53.690298 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d46f4-9858-47d1-8c82-6c972cd2b278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:53.691682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d46f4-9858-47d1-8c82-6c972cd2b278 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:53.692275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d46f4-9858-47d1-8c82-6c972cd2b278 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:53.692835 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d46f4-9858-47d1-8c82-6c972cd2b278 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:53.698111 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:53.709843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d46f4-9858-47d1-8c82-6c972cd2b278 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:53.712108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d46f4-9858-47d1-8c82-6c972cd2b278 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-517d46f4-9858-47d1-8c82-6c972cd2b278 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:43:53.749215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:53.750932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:53.751911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:53.751911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:53.751911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:43:53.751911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:43:53.751911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:43:53.751911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:43:53.751911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:43:53.751911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:43:53.751911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:43:53.751911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:43:53.752579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 620/1239] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:53.758588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706926d-e177-493c-a74f-8460a54575cc None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:53.886224 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706926d-e177-493c-a74f-8460a54575cc None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:53.929734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a19f57-9a93-451a-a04e-68d3bb6fd819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:53.930409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19f57-9a93-451a-a04e-68d3bb6fd819 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:53.930749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19f57-9a93-451a-a04e-68d3bb6fd819 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:53.931050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19f57-9a93-451a-a04e-68d3bb6fd819 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:53.931984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a19f57-9a93-451a-a04e-68d3bb6fd819 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:53.932320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a19f57-9a93-451a-a04e-68d3bb6fd819 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:53.934224 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a19f57-9a93-451a-a04e-68d3bb6fd819 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:53.958687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 620/1240] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jul 19 20:43:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:53.965283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d46f4-9858-47d1-8c82-6c972cd2b278 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:54.133839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d46f4-9858-47d1-8c82-6c972cd2b278 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:54.179220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47548c-0a61-42ef-9cff-99f83aa8d996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:54.179988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47548c-0a61-42ef-9cff-99f83aa8d996 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:54.180199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47548c-0a61-42ef-9cff-99f83aa8d996 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:54.180507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47548c-0a61-42ef-9cff-99f83aa8d996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:54.275499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:54.287854 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47548c-0a61-42ef-9cff-99f83aa8d996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:54.289865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47548c-0a61-42ef-9cff-99f83aa8d996 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:54.414014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 621/1241] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:43:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:43:54.420495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a19f57-9a93-451a-a04e-68d3bb6fd819 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:54.458908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:54.568747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a19f57-9a93-451a-a04e-68d3bb6fd819 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:54.631093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115c17e-98d3-4120-b6ea-8037bf5e3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:54.632463 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115c17e-98d3-4120-b6ea-8037bf5e3754 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:54.633320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115c17e-98d3-4120-b6ea-8037bf5e3754 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:54.633320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115c17e-98d3-4120-b6ea-8037bf5e3754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:54.654233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115c17e-98d3-4120-b6ea-8037bf5e3754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:54.657989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115c17e-98d3-4120-b6ea-8037bf5e3754 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-8115c17e-98d3-4120-b6ea-8037bf5e3754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:43:54.688636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:54.690954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:54.692959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:43:54.692959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:43:54.692959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:54.692959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:43:54.695114 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 621/1242] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:43:54.696101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47548c-0a61-42ef-9cff-99f83aa8d996 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:54.843947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:54.850168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47548c-0a61-42ef-9cff-99f83aa8d996 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:54.908222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b271b-ea8f-4a8b-a4a8-ffd86a57ae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:54.908939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b271b-ea8f-4a8b-a4a8-ffd86a57ae6b None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:54.909227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b271b-ea8f-4a8b-a4a8-ffd86a57ae6b None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:54.909506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b271b-ea8f-4a8b-a4a8-ffd86a57ae6b None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:54.910274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b271b-ea8f-4a8b-a4a8-ffd86a57ae6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:54.911816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b271b-ea8f-4a8b-a4a8-ffd86a57ae6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:54.913531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b271b-ea8f-4a8b-a4a8-ffd86a57ae6b None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:54.933528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 622/1243] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:43:54.939690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115c17e-98d3-4120-b6ea-8037bf5e3754 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:55.035459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:55.069095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115c17e-98d3-4120-b6ea-8037bf5e3754 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:55.132256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0f06c-1fce-4323-a27a-f5de71959d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:55.133173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0f06c-1fce-4323-a27a-f5de71959d5b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:55.133454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0f06c-1fce-4323-a27a-f5de71959d5b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:55.133722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0f06c-1fce-4323-a27a-f5de71959d5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:55.143534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0f06c-1fce-4323-a27a-f5de71959d5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:55.146013 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0f06c-1fce-4323-a27a-f5de71959d5b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-17d0f06c-1fce-4323-a27a-f5de71959d5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:43:55.166768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:55.168144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:55.169296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:43:55.169296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:43:55.169296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:43:55.169296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:43:55.169296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:43:55.169296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:43:55.169296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:43:55.169296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:43:55.169296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 622/1244] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:43:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:43:55.175170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b271b-ea8f-4a8b-a4a8-ffd86a57ae6b None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:55.326907 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b271b-ea8f-4a8b-a4a8-ffd86a57ae6b None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:55.366263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242a133-3b12-4836-a92c-e1615c2c67b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:55.366930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242a133-3b12-4836-a92c-e1615c2c67b5 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:55.367208 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242a133-3b12-4836-a92c-e1615c2c67b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:55.367516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242a133-3b12-4836-a92c-e1615c2c67b5 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:55.368508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242a133-3b12-4836-a92c-e1615c2c67b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:55.368906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242a133-3b12-4836-a92c-e1615c2c67b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:55.370971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242a133-3b12-4836-a92c-e1615c2c67b5 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:55.394743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 623/1245] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:43:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:43:55.395880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0f06c-1fce-4323-a27a-f5de71959d5b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:55.534822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0f06c-1fce-4323-a27a-f5de71959d5b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:55.575003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b6b23-102a-4602-917c-73bd9754941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:55.575747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b6b23-102a-4602-917c-73bd9754941a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:55.576010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b6b23-102a-4602-917c-73bd9754941a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:55.576288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b6b23-102a-4602-917c-73bd9754941a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:55.577343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b6b23-102a-4602-917c-73bd9754941a None admin] RBAC: Adding query filter params (role.id=b2e0a9d2d13844e980a69f441f36d492, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=fffebfc51dba47f68263c0bcd6b18d22) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:55.577657 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b6b23-102a-4602-917c-73bd9754941a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:55.579898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b6b23-102a-4602-917c-73bd9754941a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:55.588303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 623/1246] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:43:55 2022] GET /identity/v3/role_assignments?user.id=fffebfc51dba47f68263c0bcd6b18d22&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=b2e0a9d2d13844e980a69f441f36d492 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:43:57.948341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb39d60-ba69-4c58-ba8f-22d8b9d7a418 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:57.948920 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb39d60-ba69-4c58-ba8f-22d8b9d7a418 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:57.949435 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb39d60-ba69-4c58-ba8f-22d8b9d7a418 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:57.951935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 624/1247] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:43:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:43:57.957861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319647d-829b-4855-933a-99e9ae2006dd None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:57.958351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319647d-829b-4855-933a-99e9ae2006dd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:57.958497 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319647d-829b-4855-933a-99e9ae2006dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:57.965867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:57.978832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-3319647d-829b-4855-933a-99e9ae2006dd None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:43:57.980063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3319647d-829b-4855-933a-99e9ae2006dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:43:58.063108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 624/1248] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:43:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:43:58.089040 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb39d60-ba69-4c58-ba8f-22d8b9d7a418 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:58.126969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:58.254659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb39d60-ba69-4c58-ba8f-22d8b9d7a418 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:58.303608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad4e2e-e647-4b6a-b8d8-5af40ea32027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:58.304462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad4e2e-e647-4b6a-b8d8-5af40ea32027 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:58.304754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad4e2e-e647-4b6a-b8d8-5af40ea32027 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:58.305040 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad4e2e-e647-4b6a-b8d8-5af40ea32027 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:58.318304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad4e2e-e647-4b6a-b8d8-5af40ea32027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-fcad4e2e-e647-4b6a-b8d8-5af40ea32027 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:43:58.329880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:58.339772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:43:58.341573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 625/1249] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:43:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:58.343315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319647d-829b-4855-933a-99e9ae2006dd None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:58.456292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319647d-829b-4855-933a-99e9ae2006dd None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:58.491262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:58.496261 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aa0c8-8902-4630-9b9d-ae0dec417ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:58.497099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa0c8-8902-4630-9b9d-ae0dec417ff1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:58.497384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa0c8-8902-4630-9b9d-ae0dec417ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:58.497659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa0c8-8902-4630-9b9d-ae0dec417ff1 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:58.498525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aa0c8-8902-4630-9b9d-ae0dec417ff1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:58.498836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aa0c8-8902-4630-9b9d-ae0dec417ff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:58.500234 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aa0c8-8902-4630-9b9d-ae0dec417ff1 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:58.510222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 625/1250] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:43:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:58.516036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad4e2e-e647-4b6a-b8d8-5af40ea32027 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:58.617698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:58.651250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad4e2e-e647-4b6a-b8d8-5af40ea32027 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:58.700013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a5977-3866-4b25-9e81-c95154744310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:58.700953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a5977-3866-4b25-9e81-c95154744310 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:58.701263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a5977-3866-4b25-9e81-c95154744310 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:58.701579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a5977-3866-4b25-9e81-c95154744310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:58.724964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a5977-3866-4b25-9e81-c95154744310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:58.726954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a5977-3866-4b25-9e81-c95154744310 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-072a5977-3866-4b25-9e81-c95154744310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:43:58.748143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:58.749313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:58.750526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:43:58.750526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:43:58.750526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:43:58.750526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:43:58.750526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 626/1251] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:58.756010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aa0c8-8902-4630-9b9d-ae0dec417ff1 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:58.873009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aa0c8-8902-4630-9b9d-ae0dec417ff1 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:58.917429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17402179-8de9-42b4-8ee4-d52a7ec67d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:58.918111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17402179-8de9-42b4-8ee4-d52a7ec67d4a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:58.918404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17402179-8de9-42b4-8ee4-d52a7ec67d4a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:58.918690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17402179-8de9-42b4-8ee4-d52a7ec67d4a None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:58.919510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17402179-8de9-42b4-8ee4-d52a7ec67d4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:58.919826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17402179-8de9-42b4-8ee4-d52a7ec67d4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:58.921246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17402179-8de9-42b4-8ee4-d52a7ec67d4a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:58.940372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 626/1252] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:58.948545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a5977-3866-4b25-9e81-c95154744310 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:59.130974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a5977-3866-4b25-9e81-c95154744310 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:59.175943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:59.187584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb8b75-9535-4378-a685-fba47b38ccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:59.188722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb8b75-9535-4378-a685-fba47b38ccd9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:59.189211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb8b75-9535-4378-a685-fba47b38ccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:59.192429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb8b75-9535-4378-a685-fba47b38ccd9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:59.199442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb8b75-9535-4378-a685-fba47b38ccd9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:59.201123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb8b75-9535-4378-a685-fba47b38ccd9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-91bb8b75-9535-4378-a685-fba47b38ccd9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:43:59.220267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:43:59.221474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:43:59.222467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:43:59.222467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:43:59.222467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:43:59.222467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:43:59.222467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:43:59.222467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:43:59.222467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:43:59.222467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:43:59.222467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:43:59.222467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:43:59.222467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 627/1253] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:43:59.228003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17402179-8de9-42b4-8ee4-d52a7ec67d4a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:59.306888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:59.370793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17402179-8de9-42b4-8ee4-d52a7ec67d4a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:59.420677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f9a68-9f85-4a49-921d-ff089b1603d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:59.421481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f9a68-9f85-4a49-921d-ff089b1603d7 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:59.421805 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f9a68-9f85-4a49-921d-ff089b1603d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:59.422076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f9a68-9f85-4a49-921d-ff089b1603d7 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:59.422929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f9a68-9f85-4a49-921d-ff089b1603d7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:43:59.423235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f9a68-9f85-4a49-921d-ff089b1603d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:59.424700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f9a68-9f85-4a49-921d-ff089b1603d7 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:59.445879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 627/1254] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jul 19 20:43:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:43:59.452449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb8b75-9535-4378-a685-fba47b38ccd9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:59.570821 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb8b75-9535-4378-a685-fba47b38ccd9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:59.614048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122985ef-7d67-4851-995a-6918cfbbbfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:43:59.614709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122985ef-7d67-4851-995a-6918cfbbbfc9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:43:59.614994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122985ef-7d67-4851-995a-6918cfbbbfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:43:59.615303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122985ef-7d67-4851-995a-6918cfbbbfc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:43:59.685426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:43:59.701552 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122985ef-7d67-4851-995a-6918cfbbbfc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:43:59.703720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122985ef-7d67-4851-995a-6918cfbbbfc9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:43:59.824295 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 628/1255] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:43:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:43:59.830938 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f9a68-9f85-4a49-921d-ff089b1603d7 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:43:59.961440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f9a68-9f85-4a49-921d-ff089b1603d7 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:43:59.978796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:00.008869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe38f7f-445a-4d79-a918-66e2cc903c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:00.009962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe38f7f-445a-4d79-a918-66e2cc903c5e None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:00.010260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe38f7f-445a-4d79-a918-66e2cc903c5e None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:00.010538 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe38f7f-445a-4d79-a918-66e2cc903c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:00.031483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe38f7f-445a-4d79-a918-66e2cc903c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:00.034756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe38f7f-445a-4d79-a918-66e2cc903c5e None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-dfe38f7f-445a-4d79-a918-66e2cc903c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:44:00.060385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:00.061371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:00.063152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:44:00.063152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:44:00.063152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:44:00.063152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:00.063152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 628/1256] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:00.068913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122985ef-7d67-4851-995a-6918cfbbbfc9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:00.173366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:00.189349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122985ef-7d67-4851-995a-6918cfbbbfc9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:00.243824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9ba8e-4e49-437c-b851-ae5f1972df30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:00.244521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9ba8e-4e49-437c-b851-ae5f1972df30 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:00.244802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9ba8e-4e49-437c-b851-ae5f1972df30 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:00.245075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9ba8e-4e49-437c-b851-ae5f1972df30 None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:00.245804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9ba8e-4e49-437c-b851-ae5f1972df30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:00.246146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9ba8e-4e49-437c-b851-ae5f1972df30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:00.247669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9ba8e-4e49-437c-b851-ae5f1972df30 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:00.263502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 629/1257] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:00.270526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe38f7f-445a-4d79-a918-66e2cc903c5e None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:00.393226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe38f7f-445a-4d79-a918-66e2cc903c5e None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:00.441621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d77796-ddbf-4b95-bda7-1141b6f6550b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:00.442438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d77796-ddbf-4b95-bda7-1141b6f6550b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:00.442733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d77796-ddbf-4b95-bda7-1141b6f6550b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:00.443100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d77796-ddbf-4b95-bda7-1141b6f6550b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:00.450886 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d77796-ddbf-4b95-bda7-1141b6f6550b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:00.452560 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d77796-ddbf-4b95-bda7-1141b6f6550b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-99d77796-ddbf-4b95-bda7-1141b6f6550b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:44:00.467130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:00.468195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:00.469200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:44:00.469200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:44:00.469200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:44:00.469200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:44:00.469200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:44:00.469200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:44:00.469200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:44:00.469200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:00.469200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 629/1258] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:44:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:00.478140 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9ba8e-4e49-437c-b851-ae5f1972df30 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:00.591895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9ba8e-4e49-437c-b851-ae5f1972df30 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:00.629758 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6ba73-8ef3-43c8-8c23-83e267bbc530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:00.630423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6ba73-8ef3-43c8-8c23-83e267bbc530 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:00.630705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6ba73-8ef3-43c8-8c23-83e267bbc530 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:00.630976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6ba73-8ef3-43c8-8c23-83e267bbc530 None admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:00.631760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6ba73-8ef3-43c8-8c23-83e267bbc530 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:00.632069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6ba73-8ef3-43c8-8c23-83e267bbc530 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:00.633670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6ba73-8ef3-43c8-8c23-83e267bbc530 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:00.648544 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 630/1259] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:44:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:00.655114 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d77796-ddbf-4b95-bda7-1141b6f6550b None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:00.691245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:00.786925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d77796-ddbf-4b95-bda7-1141b6f6550b None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:00.828560 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059bb47-342a-4b87-b8e5-a1601a923134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:00.829371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059bb47-342a-4b87-b8e5-a1601a923134 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:00.829693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059bb47-342a-4b87-b8e5-a1601a923134 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:00.830000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059bb47-342a-4b87-b8e5-a1601a923134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:00.830849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059bb47-342a-4b87-b8e5-a1601a923134 None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=fffebfc51dba47f68263c0bcd6b18d22) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:00.831262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059bb47-342a-4b87-b8e5-a1601a923134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:00.832794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059bb47-342a-4b87-b8e5-a1601a923134 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:00.839755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 630/1260] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:44:00 2022] GET /identity/v3/role_assignments?user.id=fffebfc51dba47f68263c0bcd6b18d22&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:02.950704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b76842-e848-44ba-9ac4-dba203bb73a6 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:02.951241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b76842-e848-44ba-9ac4-dba203bb73a6 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:02.951241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b76842-e848-44ba-9ac4-dba203bb73a6 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:02.952711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 631/1261] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:44:02.958569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff422a5-7ad0-406d-8d4b-77581f7ebdbb None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:02.959491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff422a5-7ad0-406d-8d4b-77581f7ebdbb None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:02.959491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff422a5-7ad0-406d-8d4b-77581f7ebdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:02.976430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-6ff422a5-7ad0-406d-8d4b-77581f7ebdbb None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:44:02.977719 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6ff422a5-7ad0-406d-8d4b-77581f7ebdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:44:03.048121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 631/1262] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:44:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:44:03.071736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b76842-e848-44ba-9ac4-dba203bb73a6 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:03.083809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:03.179977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b76842-e848-44ba-9ac4-dba203bb73a6 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:03.218335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0072e7f-31d7-47ab-9f6b-26aadd6c5486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:03.219302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0072e7f-31d7-47ab-9f6b-26aadd6c5486 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:03.219761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0072e7f-31d7-47ab-9f6b-26aadd6c5486 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:03.220056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0072e7f-31d7-47ab-9f6b-26aadd6c5486 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:03.231338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0072e7f-31d7-47ab-9f6b-26aadd6c5486 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-c0072e7f-31d7-47ab-9f6b-26aadd6c5486 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:44:03.239613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:03.240819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:44:03.241750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 632/1263] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:44:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:03.247278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff422a5-7ad0-406d-8d4b-77581f7ebdbb None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:03.266806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:03.350474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff422a5-7ad0-406d-8d4b-77581f7ebdbb None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:03.386540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d8aef-56ff-4759-b0c8-695eb0c5da62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:03.387202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d8aef-56ff-4759-b0c8-695eb0c5da62 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:03.387485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d8aef-56ff-4759-b0c8-695eb0c5da62 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:03.387761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d8aef-56ff-4759-b0c8-695eb0c5da62 None admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:03.388531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d8aef-56ff-4759-b0c8-695eb0c5da62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:03.388837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d8aef-56ff-4759-b0c8-695eb0c5da62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:03.390248 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d8aef-56ff-4759-b0c8-695eb0c5da62 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:03.398736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 632/1264] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:44:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:44:03.404949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0072e7f-31d7-47ab-9f6b-26aadd6c5486 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:03.474771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:03.511037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0072e7f-31d7-47ab-9f6b-26aadd6c5486 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:03.545120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef28a5-d157-49f0-81fe-5738fab4a076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:03.545899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef28a5-d157-49f0-81fe-5738fab4a076 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:03.546179 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef28a5-d157-49f0-81fe-5738fab4a076 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:03.547061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef28a5-d157-49f0-81fe-5738fab4a076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:03.558621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef28a5-d157-49f0-81fe-5738fab4a076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:03.560071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef28a5-d157-49f0-81fe-5738fab4a076 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-c1ef28a5-d157-49f0-81fe-5738fab4a076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:44:03.578046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:03.579129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:03.580303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:44:03.580303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:44:03.580303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:44:03.580303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:44:03.580303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 633/1265] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:44:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:44:03.585525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d8aef-56ff-4759-b0c8-695eb0c5da62 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:03.674840 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:03.696913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d8aef-56ff-4759-b0c8-695eb0c5da62 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:03.740654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b33f1-758c-4b35-9e09-5e1595ff1ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:03.741252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b33f1-758c-4b35-9e09-5e1595ff1ff1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:03.741539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b33f1-758c-4b35-9e09-5e1595ff1ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:03.741819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b33f1-758c-4b35-9e09-5e1595ff1ff1 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:03.742545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b33f1-758c-4b35-9e09-5e1595ff1ff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:03.742855 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b33f1-758c-4b35-9e09-5e1595ff1ff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:03.744230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b33f1-758c-4b35-9e09-5e1595ff1ff1 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:03.756812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 633/1266] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:44:03.762943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef28a5-d157-49f0-81fe-5738fab4a076 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:03.864051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef28a5-d157-49f0-81fe-5738fab4a076 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:03.885332 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:03.911372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a8885-f9d7-484b-9cd3-101ebf8c7416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:03.912190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a8885-f9d7-484b-9cd3-101ebf8c7416 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:03.912504 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a8885-f9d7-484b-9cd3-101ebf8c7416 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:03.912785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a8885-f9d7-484b-9cd3-101ebf8c7416 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:03.920804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a8885-f9d7-484b-9cd3-101ebf8c7416 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:03.922263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a8885-f9d7-484b-9cd3-101ebf8c7416 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-c23a8885-f9d7-484b-9cd3-101ebf8c7416 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:44:03.937541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:03.938777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:03.939778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:03.939778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:03.939778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:44:03.939778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:44:03.939778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:44:03.939778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:44:03.939778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:44:03.939778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:44:03.939778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:44:03.939778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:44:03.939778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 634/1267] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:44:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:03.945701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b33f1-758c-4b35-9e09-5e1595ff1ff1 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:04.054819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b33f1-758c-4b35-9e09-5e1595ff1ff1 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:04.093985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5da8b4-43dc-462a-85fc-a6a80ded99b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:04.094663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da8b4-43dc-462a-85fc-a6a80ded99b8 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:04.094948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da8b4-43dc-462a-85fc-a6a80ded99b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:04.095229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da8b4-43dc-462a-85fc-a6a80ded99b8 None admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:04.096175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5da8b4-43dc-462a-85fc-a6a80ded99b8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:04.096523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5da8b4-43dc-462a-85fc-a6a80ded99b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:04.098366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5da8b4-43dc-462a-85fc-a6a80ded99b8 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:04.108077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:04.118404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 634/1268] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jul 19 20:44:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:44:04.124899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a8885-f9d7-484b-9cd3-101ebf8c7416 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:04.242878 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a8885-f9d7-484b-9cd3-101ebf8c7416 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:04.278499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f621462-b4c8-4aac-ab8b-679412899a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:04.279290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f621462-b4c8-4aac-ab8b-679412899a14 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:04.279571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f621462-b4c8-4aac-ab8b-679412899a14 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:04.279845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f621462-b4c8-4aac-ab8b-679412899a14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:04.291709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f621462-b4c8-4aac-ab8b-679412899a14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:04.293233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f621462-b4c8-4aac-ab8b-679412899a14 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-4f621462-b4c8-4aac-ab8b-679412899a14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:44:04.311061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:04.312886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:04.314027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:44:04.314027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:44:04.314027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:44:04.314027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:44:04.314027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 635/1269] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:44:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:04.323156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5da8b4-43dc-462a-85fc-a6a80ded99b8 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:04.448052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5da8b4-43dc-462a-85fc-a6a80ded99b8 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:04.482420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5011ee-2833-42a8-885e-1bcc25797b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:04.483089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5011ee-2833-42a8-885e-1bcc25797b93 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:04.483375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5011ee-2833-42a8-885e-1bcc25797b93 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:04.483653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5011ee-2833-42a8-885e-1bcc25797b93 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:04.484436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5011ee-2833-42a8-885e-1bcc25797b93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:04.484749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5011ee-2833-42a8-885e-1bcc25797b93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:04.486117 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5011ee-2833-42a8-885e-1bcc25797b93 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:04.499093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 635/1270] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:04.505379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f621462-b4c8-4aac-ab8b-679412899a14 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:04.538223 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:04.616234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f621462-b4c8-4aac-ab8b-679412899a14 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:04.653846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75fa57-06c8-4219-b49d-8e30ef0651b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:04.654679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75fa57-06c8-4219-b49d-8e30ef0651b9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:04.655036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75fa57-06c8-4219-b49d-8e30ef0651b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:04.655314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75fa57-06c8-4219-b49d-8e30ef0651b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:04.665645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75fa57-06c8-4219-b49d-8e30ef0651b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:04.667364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75fa57-06c8-4219-b49d-8e30ef0651b9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-cc75fa57-06c8-4219-b49d-8e30ef0651b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:44:04.685997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:04.687717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:04.688955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:44:04.688955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:44:04.688955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:44:04.688955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:44:04.688955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:44:04.688955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:44:04.688955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:44:04.688955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:44:04.688955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 636/1271] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:44:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:44:04.696909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5011ee-2833-42a8-885e-1bcc25797b93 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:04.757144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:04.808157 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5011ee-2833-42a8-885e-1bcc25797b93 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:04.844389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687877d6-8dfb-43d2-9643-2254dfad48af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:04.845089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687877d6-8dfb-43d2-9643-2254dfad48af None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:04.845370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687877d6-8dfb-43d2-9643-2254dfad48af None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:04.845643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687877d6-8dfb-43d2-9643-2254dfad48af None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:04.846434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687877d6-8dfb-43d2-9643-2254dfad48af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:04.846741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687877d6-8dfb-43d2-9643-2254dfad48af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:04.848182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687877d6-8dfb-43d2-9643-2254dfad48af None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:04.862243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 636/1272] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:44:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:04.868924 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75fa57-06c8-4219-b49d-8e30ef0651b9 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:04.966009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:04.975360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75fa57-06c8-4219-b49d-8e30ef0651b9 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:05.013492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2874b2-e257-4a6d-b619-65b45f576a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:05.014379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2874b2-e257-4a6d-b619-65b45f576a57 None admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:05.014662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2874b2-e257-4a6d-b619-65b45f576a57 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:05.014938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2874b2-e257-4a6d-b619-65b45f576a57 None admin] PATH_INFO: `/v3/projects/1aba35520c454245854855df2e5b42f9/users/fffebfc51dba47f68263c0bcd6b18d22/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:05.016941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2874b2-e257-4a6d-b619-65b45f576a57 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aba35520c454245854855df2e5b42f9, user_id=fffebfc51dba47f68263c0bcd6b18d22, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:05.018376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2874b2-e257-4a6d-b619-65b45f576a57 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:05.033103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 637/1273] 10.222.0.85 () {58 vars in 1439 bytes} [Tue Jul 19 20:44:04 2022] PUT /identity/v3/projects/1aba35520c454245854855df2e5b42f9/users/fffebfc51dba47f68263c0bcd6b18d22/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:44:07.071637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de4e5a-9270-433c-a561-b730e9b5723f None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:07.072181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de4e5a-9270-433c-a561-b730e9b5723f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:07.072272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de4e5a-9270-433c-a561-b730e9b5723f None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:07.074536 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 637/1274] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:44:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:44:07.082046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44246966-b09e-4e50-aa61-fbf47a90d88c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:07.082541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44246966-b09e-4e50-aa61-fbf47a90d88c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:07.082655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44246966-b09e-4e50-aa61-fbf47a90d88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:07.103108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-44246966-b09e-4e50-aa61-fbf47a90d88c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:44:07.103108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-44246966-b09e-4e50-aa61-fbf47a90d88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:44:07.187102 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 638/1275] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:44:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 20:44:07.212073 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de4e5a-9270-433c-a561-b730e9b5723f None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:07.331122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de4e5a-9270-433c-a561-b730e9b5723f None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:07.351555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:07.373472 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad382884-b486-47e6-b77d-4938f68d119a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:07.374280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad382884-b486-47e6-b77d-4938f68d119a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:07.375050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad382884-b486-47e6-b77d-4938f68d119a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:07.375311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad382884-b486-47e6-b77d-4938f68d119a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:07.387364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad382884-b486-47e6-b77d-4938f68d119a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-ad382884-b486-47e6-b77d-4938f68d119a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 20:44:07.395766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:07.397048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:07.402963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 638/1276] 10.222.0.85 () {56 vars in 1096 bytes} [Tue Jul 19 20:44:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:07.403926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44246966-b09e-4e50-aa61-fbf47a90d88c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:07.505065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:07.525707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44246966-b09e-4e50-aa61-fbf47a90d88c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:07.562405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26660d1-bd55-47b7-8cfe-737da88a40fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:07.563086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26660d1-bd55-47b7-8cfe-737da88a40fa None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:07.563366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26660d1-bd55-47b7-8cfe-737da88a40fa None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:07.563640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26660d1-bd55-47b7-8cfe-737da88a40fa None admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:07.564406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26660d1-bd55-47b7-8cfe-737da88a40fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:07.564716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26660d1-bd55-47b7-8cfe-737da88a40fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:07.566084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26660d1-bd55-47b7-8cfe-737da88a40fa None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:07.575245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 639/1277] 10.222.0.85 () {56 vars in 1099 bytes} [Tue Jul 19 20:44:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:07.584880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad382884-b486-47e6-b77d-4938f68d119a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:07.723072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad382884-b486-47e6-b77d-4938f68d119a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:07.775457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07b12c-bdd7-49a0-a66b-0661ed59f9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:07.776379 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07b12c-bdd7-49a0-a66b-0661ed59f9b1 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:07.776671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07b12c-bdd7-49a0-a66b-0661ed59f9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:07.776953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07b12c-bdd7-49a0-a66b-0661ed59f9b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:07.789197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07b12c-bdd7-49a0-a66b-0661ed59f9b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:07.790790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07b12c-bdd7-49a0-a66b-0661ed59f9b1 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-6f07b12c-bdd7-49a0-a66b-0661ed59f9b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:44:07.810425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:07.812698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:07.814658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:44:07.814658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:44:07.814658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:44:07.814658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:07.814658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 639/1278] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:07.820509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26660d1-bd55-47b7-8cfe-737da88a40fa None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:07.948100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26660d1-bd55-47b7-8cfe-737da88a40fa None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:07.985146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26554ef-3356-4111-9054-0a7800bb3dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:07.985806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26554ef-3356-4111-9054-0a7800bb3dca None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:07.986102 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26554ef-3356-4111-9054-0a7800bb3dca None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:07.986376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26554ef-3356-4111-9054-0a7800bb3dca None admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:07.987092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26554ef-3356-4111-9054-0a7800bb3dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:07.987434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26554ef-3356-4111-9054-0a7800bb3dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:07.988899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26554ef-3356-4111-9054-0a7800bb3dca None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:08.002443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 640/1279] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:08.008397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07b12c-bdd7-49a0-a66b-0661ed59f9b1 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:08.042022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:08.118251 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07b12c-bdd7-49a0-a66b-0661ed59f9b1 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:08.157015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239b0be-f079-428c-bcf1-2f6b1d3b2fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:08.157819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239b0be-f079-428c-bcf1-2f6b1d3b2fb4 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:08.158098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239b0be-f079-428c-bcf1-2f6b1d3b2fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:08.158370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239b0be-f079-428c-bcf1-2f6b1d3b2fb4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:08.166722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239b0be-f079-428c-bcf1-2f6b1d3b2fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:08.168159 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239b0be-f079-428c-bcf1-2f6b1d3b2fb4 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-0239b0be-f079-428c-bcf1-2f6b1d3b2fb4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 20:44:08.183854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:08.185215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:08.186400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:08.186400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:08.186400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 20:44:08.186400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 20:44:08.186400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 20:44:08.186400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 20:44:08.186400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 20:44:08.186400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 20:44:08.186400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 20:44:08.186400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:08.186400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 640/1280] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:44:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 20:44:08.191254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26554ef-3356-4111-9054-0a7800bb3dca None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:08.195490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:08.309451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26554ef-3356-4111-9054-0a7800bb3dca None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:08.349379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d7a7a-b11c-4536-87fc-4a70b65254d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:08.350053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d7a7a-b11c-4536-87fc-4a70b65254d8 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:08.350332 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d7a7a-b11c-4536-87fc-4a70b65254d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:08.350606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d7a7a-b11c-4536-87fc-4a70b65254d8 None admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:08.351441 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d7a7a-b11c-4536-87fc-4a70b65254d8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:08.351741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d7a7a-b11c-4536-87fc-4a70b65254d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:08.353201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d7a7a-b11c-4536-87fc-4a70b65254d8 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:08.371807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 641/1281] 10.222.0.85 () {56 vars in 1143 bytes} [Tue Jul 19 20:44:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:44:08.377723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239b0be-f079-428c-bcf1-2f6b1d3b2fb4 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:08.482840 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:08.493572 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239b0be-f079-428c-bcf1-2f6b1d3b2fb4 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:08.531637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc855e-bd5a-4029-970e-ea5b3a85687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:08.532296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc855e-bd5a-4029-970e-ea5b3a85687a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:08.532611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc855e-bd5a-4029-970e-ea5b3a85687a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:08.532882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc855e-bd5a-4029-970e-ea5b3a85687a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:08.603402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc855e-bd5a-4029-970e-ea5b3a85687a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:08.604943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc855e-bd5a-4029-970e-ea5b3a85687a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:08.710541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:08.722866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 641/1282] 10.222.0.85 () {58 vars in 1282 bytes} [Tue Jul 19 20:44:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 20:44:08.728835 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d7a7a-b11c-4536-87fc-4a70b65254d8 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:08.794193 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:08.834207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d7a7a-b11c-4536-87fc-4a70b65254d8 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:08.868138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046807c-beb8-43d4-bbb3-0b149ceb0043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:08.868962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046807c-beb8-43d4-bbb3-0b149ceb0043 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:08.869977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046807c-beb8-43d4-bbb3-0b149ceb0043 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:08.869977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046807c-beb8-43d4-bbb3-0b149ceb0043 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:08.881754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046807c-beb8-43d4-bbb3-0b149ceb0043 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:08.883224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046807c-beb8-43d4-bbb3-0b149ceb0043 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-9046807c-beb8-43d4-bbb3-0b149ceb0043 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 20:44:08.902001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:08.903056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:08.904120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 20:44:08.904120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 20:44:08.904120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 20:44:08.904120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:44:08.904120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 642/1283] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:08.909519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc855e-bd5a-4029-970e-ea5b3a85687a None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:09.014882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc855e-bd5a-4029-970e-ea5b3a85687a None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:09.052447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c301-d667-4e91-bedd-1ad0271ce611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:09.053040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c301-d667-4e91-bedd-1ad0271ce611 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:09.053327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c301-d667-4e91-bedd-1ad0271ce611 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:09.053604 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c301-d667-4e91-bedd-1ad0271ce611 None admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:09.054609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44c301-d667-4e91-bedd-1ad0271ce611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:09.054928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44c301-d667-4e91-bedd-1ad0271ce611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:09.056315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44c301-d667-4e91-bedd-1ad0271ce611 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:09.069885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 642/1284] 10.222.0.85 () {56 vars in 1109 bytes} [Tue Jul 19 20:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:44:09.077803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046807c-beb8-43d4-bbb3-0b149ceb0043 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:09.185104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046807c-beb8-43d4-bbb3-0b149ceb0043 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:09.200778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:09.224983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7c59f-0ded-445f-a1a6-21c09f27a8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:09.225794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c59f-0ded-445f-a1a6-21c09f27a8ff None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:09.226072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c59f-0ded-445f-a1a6-21c09f27a8ff None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:09.226342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c59f-0ded-445f-a1a6-21c09f27a8ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:09.233889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7c59f-0ded-445f-a1a6-21c09f27a8ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:09.235438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7c59f-0ded-445f-a1a6-21c09f27a8ff None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application [None req-71d7c59f-0ded-445f-a1a6-21c09f27a8ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 20:44:09.249668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:09.250842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:09.251951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 20:44:09.251951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 20:44:09.251951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 20:44:09.251951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 20:44:09.251951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 20:44:09.251951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 20:44:09.251951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 20:44:09.251951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: ERROR keystone.server.flask.application Jul 19 20:44:09.251951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 643/1285] 10.222.0.85 () {56 vars in 1111 bytes} [Tue Jul 19 20:44:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:09.257282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c301-d667-4e91-bedd-1ad0271ce611 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:09.312771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:09.380626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c301-d667-4e91-bedd-1ad0271ce611 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:09.419593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dc148-a801-4f76-b5d0-4127b819e1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:09.420263 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dc148-a801-4f76-b5d0-4127b819e1c5 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:09.420576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dc148-a801-4f76-b5d0-4127b819e1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:09.420856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dc148-a801-4f76-b5d0-4127b819e1c5 None admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:09.421681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dc148-a801-4f76-b5d0-4127b819e1c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:09.421993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dc148-a801-4f76-b5d0-4127b819e1c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:09.423427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dc148-a801-4f76-b5d0-4127b819e1c5 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:09.438034 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 643/1286] 10.222.0.85 () {56 vars in 1148 bytes} [Tue Jul 19 20:44:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:09.444444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7c59f-0ded-445f-a1a6-21c09f27a8ff None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:09.569587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7c59f-0ded-445f-a1a6-21c09f27a8ff None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:09.604505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef0830-ad76-4668-a000-e96e7fe11175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:09.605166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef0830-ad76-4668-a000-e96e7fe11175 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:09.605445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef0830-ad76-4668-a000-e96e7fe11175 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:09.605720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef0830-ad76-4668-a000-e96e7fe11175 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:09.606583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef0830-ad76-4668-a000-e96e7fe11175 None admin] RBAC: Adding query filter params (role.id=441f27d47b2f45e3a60f724e615a42f7, scope.project.id=1aba35520c454245854855df2e5b42f9, user.id=fffebfc51dba47f68263c0bcd6b18d22) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:09.606889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef0830-ad76-4668-a000-e96e7fe11175 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:09.608363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef0830-ad76-4668-a000-e96e7fe11175 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:09.614756 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 644/1287] 10.222.0.85 () {56 vars in 1374 bytes} [Tue Jul 19 20:44:09 2022] GET /identity/v3/role_assignments?user.id=fffebfc51dba47f68263c0bcd6b18d22&scope.project.id=1aba35520c454245854855df2e5b42f9&role.id=441f27d47b2f45e3a60f724e615a42f7 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:44:11.710782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7065051-7dd2-477c-8e45-2ac0b25e94a4 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:11.711436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7065051-7dd2-477c-8e45-2ac0b25e94a4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:11.711436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7065051-7dd2-477c-8e45-2ac0b25e94a4 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:11.712666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 644/1288] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:44:11.718816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60fbe4-0650-4229-978c-d5a42edfa45a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:11.719498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60fbe4-0650-4229-978c-d5a42edfa45a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:11.719498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60fbe4-0650-4229-978c-d5a42edfa45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:11.742201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-8e60fbe4-0650-4229-978c-d5a42edfa45a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:44:11.743426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8e60fbe4-0650-4229-978c-d5a42edfa45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:44:11.780418 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:11.826602 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 645/1289] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:44:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:44:11.852788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7065051-7dd2-477c-8e45-2ac0b25e94a4 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:11.969882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7065051-7dd2-477c-8e45-2ac0b25e94a4 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:11.982045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:12.008102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc12e25-cb35-4859-b649-69ffdb9ce1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:12.008949 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc12e25-cb35-4859-b649-69ffdb9ce1ff None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:12.009232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc12e25-cb35-4859-b649-69ffdb9ce1ff None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:12.009511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc12e25-cb35-4859-b649-69ffdb9ce1ff None admin] PATH_INFO: `/v3/services/placement` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:12.010180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc12e25-cb35-4859-b649-69ffdb9ce1ff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:12.011575 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc12e25-cb35-4859-b649-69ffdb9ce1ff None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-1fc12e25-cb35-4859-b649-69ffdb9ce1ff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:12.024476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:12.025844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:12.027683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 645/1290] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jul 19 20:44:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:12.033122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60fbe4-0650-4229-978c-d5a42edfa45a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:12.158634 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60fbe4-0650-4229-978c-d5a42edfa45a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:12.207252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010418c-d327-4f01-9ab0-b40831550842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:12.208240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010418c-d327-4f01-9ab0-b40831550842 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:12.208698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010418c-d327-4f01-9ab0-b40831550842 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:12.209114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010418c-d327-4f01-9ab0-b40831550842 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:12.210206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010418c-d327-4f01-9ab0-b40831550842 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:12.210692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010418c-d327-4f01-9ab0-b40831550842 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:12.212931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010418c-d327-4f01-9ab0-b40831550842 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:12.221942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 646/1291] 10.222.0.85 () {56 vars in 1116 bytes} [Tue Jul 19 20:44:12 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:44:12.228580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc12e25-cb35-4859-b649-69ffdb9ce1ff None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:12.344944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc12e25-cb35-4859-b649-69ffdb9ce1ff None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:12.388566 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dd0b5-1008-4171-8bbe-7082b2a3984a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:12.389241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd0b5-1008-4171-8bbe-7082b2a3984a None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:12.389518 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd0b5-1008-4171-8bbe-7082b2a3984a None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:12.389796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd0b5-1008-4171-8bbe-7082b2a3984a None admin] PATH_INFO: `/v3/services` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:12.391034 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dd0b5-1008-4171-8bbe-7082b2a3984a None admin] RBAC: Adding query filter params (type=placement) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:12.391339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dd0b5-1008-4171-8bbe-7082b2a3984a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:12.392786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dd0b5-1008-4171-8bbe-7082b2a3984a None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:12.398754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 646/1292] 10.222.0.85 () {56 vars in 1116 bytes} [Tue Jul 19 20:44:12 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:14.521249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b3bb-508e-41f0-b7ec-2fe6df2cc1ec None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:14.521777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b3bb-508e-41f0-b7ec-2fe6df2cc1ec None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:14.521777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b3bb-508e-41f0-b7ec-2fe6df2cc1ec None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:14.523109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 647/1293] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:44:14.529032 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9aaa7-ee0c-4bd6-8dbd-110655257252 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:14.529550 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9aaa7-ee0c-4bd6-8dbd-110655257252 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:14.529667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9aaa7-ee0c-4bd6-8dbd-110655257252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:14.547018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-65e9aaa7-ee0c-4bd6-8dbd-110655257252 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:44:14.548227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-65e9aaa7-ee0c-4bd6-8dbd-110655257252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:44:14.577870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:14.617043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 647/1294] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:44:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:44:14.640775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1b3bb-508e-41f0-b7ec-2fe6df2cc1ec None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:14.677678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:14.770659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1b3bb-508e-41f0-b7ec-2fe6df2cc1ec None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:14.803992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfe727-6592-4f26-8202-776e60ecbfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:14.804591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfe727-6592-4f26-8202-776e60ecbfd8 None admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:14.804877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfe727-6592-4f26-8202-776e60ecbfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:14.805155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfe727-6592-4f26-8202-776e60ecbfd8 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:14.805879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfe727-6592-4f26-8202-776e60ecbfd8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:14.807251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dfe727-6592-4f26-8202-776e60ecbfd8 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:14.821113 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 648/1295] 10.222.0.85 () {60 vars in 1141 bytes} [Tue Jul 19 20:44:14 2022] POST /identity/v3/services => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:44:16.849270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd57400-773d-48ac-ab63-8abd7eb4079d None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:16.849831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd57400-773d-48ac-ab63-8abd7eb4079d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:16.849831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd57400-773d-48ac-ab63-8abd7eb4079d None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:16.851208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 648/1296] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:44:16.856438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c395b71-8bea-437c-af27-536577be1b5c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:16.856861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c395b71-8bea-437c-af27-536577be1b5c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:16.857005 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c395b71-8bea-437c-af27-536577be1b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:16.873330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-8c395b71-8bea-437c-af27-536577be1b5c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:44:16.874522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8c395b71-8bea-437c-af27-536577be1b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:44:16.945230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 649/1297] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:44:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:44:16.969594 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd57400-773d-48ac-ab63-8abd7eb4079d None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:17.071558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd57400-773d-48ac-ab63-8abd7eb4079d None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:17.104716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7d6b6-543d-4185-8f84-b705e9dd8a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:17.105471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7d6b6-543d-4185-8f84-b705e9dd8a6d None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:17.105757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7d6b6-543d-4185-8f84-b705e9dd8a6d None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:17.106037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7d6b6-543d-4185-8f84-b705e9dd8a6d None admin] PATH_INFO: `/v3/services/placement` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:17.106692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7d6b6-543d-4185-8f84-b705e9dd8a6d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:17.108051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7d6b6-543d-4185-8f84-b705e9dd8a6d None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-04a7d6b6-543d-4185-8f84-b705e9dd8a6d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:17.119838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:17.121047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:17.122035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 649/1298] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jul 19 20:44:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:17.127520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c395b71-8bea-437c-af27-536577be1b5c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:17.146114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:17.228559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c395b71-8bea-437c-af27-536577be1b5c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:17.262246 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a08db-0e22-423b-bbfe-7d25e956d488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:17.262910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a08db-0e22-423b-bbfe-7d25e956d488 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:17.263195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a08db-0e22-423b-bbfe-7d25e956d488 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:17.263473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a08db-0e22-423b-bbfe-7d25e956d488 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:17.264175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a08db-0e22-423b-bbfe-7d25e956d488 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:17.264521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a08db-0e22-423b-bbfe-7d25e956d488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:17.266499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a08db-0e22-423b-bbfe-7d25e956d488 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:17.271582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 650/1299] 10.222.0.85 () {56 vars in 1116 bytes} [Tue Jul 19 20:44:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:17.277552 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7d6b6-543d-4185-8f84-b705e9dd8a6d None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:17.308663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:17.376660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7d6b6-543d-4185-8f84-b705e9dd8a6d None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:17.409237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c91c2-ecce-4d0b-b7e1-fb13ef57253b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:17.409888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c91c2-ecce-4d0b-b7e1-fb13ef57253b None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:17.410174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c91c2-ecce-4d0b-b7e1-fb13ef57253b None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:17.410451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c91c2-ecce-4d0b-b7e1-fb13ef57253b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:17.411242 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c91c2-ecce-4d0b-b7e1-fb13ef57253b None admin] RBAC: Adding query filter params (interface=public, service_id=038f8e03badc4f799c31c7c6868ecb23, region_id=RegionOne) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:17.411546 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c91c2-ecce-4d0b-b7e1-fb13ef57253b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:17.412937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c91c2-ecce-4d0b-b7e1-fb13ef57253b None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:17.417908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 650/1300] 10.222.0.85 () {56 vars in 1251 bytes} [Tue Jul 19 20:44:17 2022] GET /identity/v3/endpoints?service_id=038f8e03badc4f799c31c7c6868ecb23&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:17.424198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a08db-0e22-423b-bbfe-7d25e956d488 None admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:17.503727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:17.521470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a08db-0e22-423b-bbfe-7d25e956d488 None admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:17.554912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bed7d-852a-4656-8a2f-5d1c40d820e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:17.555579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bed7d-852a-4656-8a2f-5d1c40d820e9 None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:17.555862 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bed7d-852a-4656-8a2f-5d1c40d820e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:17.556140 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bed7d-852a-4656-8a2f-5d1c40d820e9 None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:17.556862 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bed7d-852a-4656-8a2f-5d1c40d820e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:17.558483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bed7d-852a-4656-8a2f-5d1c40d820e9 None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:17.565851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 651/1301] 10.222.0.85 () {56 vars in 1087 bytes} [Tue Jul 19 20:44:17 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:44:19.622193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82026ae8-3ec6-4e02-9e43-d6092fdfaa80 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:19.622699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82026ae8-3ec6-4e02-9e43-d6092fdfaa80 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:19.622699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82026ae8-3ec6-4e02-9e43-d6092fdfaa80 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:19.624020 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 651/1302] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:44:19.629262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2036-52ff-40e7-9c91-4904db7b5ac3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:19.629642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2036-52ff-40e7-9c91-4904db7b5ac3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:19.629837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2036-52ff-40e7-9c91-4904db7b5ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:19.646784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-558d2036-52ff-40e7-9c91-4904db7b5ac3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:44:19.647958 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-558d2036-52ff-40e7-9c91-4904db7b5ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:44:19.715783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 652/1303] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:44:19 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 20:44:19.740171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82026ae8-3ec6-4e02-9e43-d6092fdfaa80 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:19.835994 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:19.843704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82026ae8-3ec6-4e02-9e43-d6092fdfaa80 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:19.876917 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deefcef-d34e-4aec-8e09-d0d4cab0d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:19.877634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefcef-d34e-4aec-8e09-d0d4cab0d0a9 None admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:19.877922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefcef-d34e-4aec-8e09-d0d4cab0d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:19.878199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deefcef-d34e-4aec-8e09-d0d4cab0d0a9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:19.878845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deefcef-d34e-4aec-8e09-d0d4cab0d0a9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:19.880212 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deefcef-d34e-4aec-8e09-d0d4cab0d0a9 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application [None req-8deefcef-d34e-4aec-8e09-d0d4cab0d0a9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 20:44:19.892555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 20:44:19.893737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application with Lock( Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self._enter() Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application return self.creator() Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application created_value = creator( Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: ERROR keystone.server.flask.application Jul 19 20:44:19.895021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 652/1304] 10.222.0.85 () {56 vars in 1117 bytes} [Tue Jul 19 20:44:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 20:44:19.899840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2036-52ff-40e7-9c91-4904db7b5ac3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:19.974768 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:19.997366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2036-52ff-40e7-9c91-4904db7b5ac3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:20.031766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33441ca1-d6e3-4220-be5f-2f10b5e7180e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:20.032449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33441ca1-d6e3-4220-be5f-2f10b5e7180e None admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:20.032732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33441ca1-d6e3-4220-be5f-2f10b5e7180e None admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:20.033006 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33441ca1-d6e3-4220-be5f-2f10b5e7180e None admin] PATH_INFO: `/v3/services` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:20.033700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33441ca1-d6e3-4220-be5f-2f10b5e7180e None admin] RBAC: Adding query filter params (name=placement) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:44:20.034005 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33441ca1-d6e3-4220-be5f-2f10b5e7180e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:20.035366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33441ca1-d6e3-4220-be5f-2f10b5e7180e None admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:20.040681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 653/1305] 10.222.0.85 () {56 vars in 1116 bytes} [Tue Jul 19 20:44:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:44:20.046448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deefcef-d34e-4aec-8e09-d0d4cab0d0a9 None admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:44:20.150755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deefcef-d34e-4aec-8e09-d0d4cab0d0a9 None admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:44:20.183387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0de943-09f9-40f1-a2a0-6a25b1dfe975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:44:20.184036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0de943-09f9-40f1-a2a0-6a25b1dfe975 None admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:44:20.184318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0de943-09f9-40f1-a2a0-6a25b1dfe975 None admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:44:20.184627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0de943-09f9-40f1-a2a0-6a25b1dfe975 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:44:20.185356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0de943-09f9-40f1-a2a0-6a25b1dfe975 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:44:20.187281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0de943-09f9-40f1-a2a0-6a25b1dfe975 None admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:44:20.206174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 653/1306] 10.222.0.85 () {60 vars in 1144 bytes} [Tue Jul 19 20:44:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:44:30.037537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=30 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:44:30.201303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:25.708559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be142a-7e1a-4dfb-a699-c38393251548 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:25.709187 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be142a-7e1a-4dfb-a699-c38393251548 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:25.709187 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be142a-7e1a-4dfb-a699-c38393251548 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:25.715898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdccb3-504b-4b5b-ba1a-52c18755449d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:25.716905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdccb3-504b-4b5b-ba1a-52c18755449d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:25.717142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdccb3-504b-4b5b-ba1a-52c18755449d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:25.720531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 654/1307] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:45:25 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:25.868387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-5acdccb3-504b-4b5b-ba1a-52c18755449d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:25.870899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-5acdccb3-504b-4b5b-ba1a-52c18755449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:26.009837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 654/1308] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:26.068994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa3fc0-f829-49dd-8102-1ddda89581c3 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:26.069384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa3fc0-f829-49dd-8102-1ddda89581c3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:26.069774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa3fc0-f829-49dd-8102-1ddda89581c3 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:26.072350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 655/1309] 10.222.0.85 () {54 vars in 965 bytes} [Tue Jul 19 20:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:26.078963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1632a17-3152-4739-928e-2246167d29fc None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:26.079270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1632a17-3152-4739-928e-2246167d29fc None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:26.079531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1632a17-3152-4739-928e-2246167d29fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:26.184381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-f1632a17-3152-4739-928e-2246167d29fc None None] MFA Rules not processed for user `5db827b3a66843749e04376559aa27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:26.186435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f1632a17-3152-4739-928e-2246167d29fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:26.223842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:26.300570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 655/1310] 10.222.0.85 () {58 vars in 1062 bytes} [Tue Jul 19 20:45:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:26.310596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa3fc0-f829-49dd-8102-1ddda89581c3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:26.312405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-14aa3fc0-f829-49dd-8102-1ddda89581c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:26.449215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa3fc0-f829-49dd-8102-1ddda89581c3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:26.534877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6438f8e-7aac-4311-9ae8-a76398c0121d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:26.535458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6438f8e-7aac-4311-9ae8-a76398c0121d service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:26.535742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6438f8e-7aac-4311-9ae8-a76398c0121d service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:26.536020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6438f8e-7aac-4311-9ae8-a76398c0121d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:26.537706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d6438f8e-7aac-4311-9ae8-a76398c0121d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:26.599228 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:26.661279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6438f8e-7aac-4311-9ae8-a76398c0121d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:26.662842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6438f8e-7aac-4311-9ae8-a76398c0121d service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:26.682943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 656/1311] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:45:29.097430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8798-cd97-416c-8b89-16502c2878e0 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:29.097889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8798-cd97-416c-8b89-16502c2878e0 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:29.098089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b8798-cd97-416c-8b89-16502c2878e0 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:29.100198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 656/1312] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:29.106805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b14b2-d3f5-44bb-ba4a-503b11b66736 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:29.107976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b14b2-d3f5-44bb-ba4a-503b11b66736 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:29.108442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b14b2-d3f5-44bb-ba4a-503b11b66736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:29.198352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-686b14b2-d3f5-44bb-ba4a-503b11b66736 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:29.199587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-686b14b2-d3f5-44bb-ba4a-503b11b66736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:29.305377 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 657/1313] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:29.361877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751c0ba-79f4-439a-83ef-626fc846f777 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:29.362302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751c0ba-79f4-439a-83ef-626fc846f777 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:29.362737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751c0ba-79f4-439a-83ef-626fc846f777 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:29.364446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 657/1314] 10.222.0.85 () {54 vars in 965 bytes} [Tue Jul 19 20:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:29.370971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280828b-078c-462f-a1be-e2c7f3e23024 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:29.371336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280828b-078c-462f-a1be-e2c7f3e23024 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:29.371685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280828b-078c-462f-a1be-e2c7f3e23024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:29.382866 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:29.457339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-8280828b-078c-462f-a1be-e2c7f3e23024 None None] MFA Rules not processed for user `5db827b3a66843749e04376559aa27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:29.458567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8280828b-078c-462f-a1be-e2c7f3e23024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:29.540687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 658/1315] 10.222.0.85 () {58 vars in 1062 bytes} [Tue Jul 19 20:45:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:29.549770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751c0ba-79f4-439a-83ef-626fc846f777 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:29.551322 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d751c0ba-79f4-439a-83ef-626fc846f777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:29.660986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751c0ba-79f4-439a-83ef-626fc846f777 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:29.737703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:29.740996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5f995-db3d-40db-9b1a-c499d4f4e694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:29.741991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5f995-db3d-40db-9b1a-c499d4f4e694 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:29.742523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5f995-db3d-40db-9b1a-c499d4f4e694 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:29.742887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5f995-db3d-40db-9b1a-c499d4f4e694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:29.744780 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-40a5f995-db3d-40db-9b1a-c499d4f4e694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:29.866951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5f995-db3d-40db-9b1a-c499d4f4e694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:29.868428 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5f995-db3d-40db-9b1a-c499d4f4e694 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:29.888736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 658/1316] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:45:32.472518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44119d-cd59-4e8d-8ed4-76b8da410462 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:32.473084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44119d-cd59-4e8d-8ed4-76b8da410462 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:32.473084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44119d-cd59-4e8d-8ed4-76b8da410462 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:32.474352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 659/1317] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:32.479998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bb2fc-71ab-4a3c-ab47-dad76c03dc31 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:32.480554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bb2fc-71ab-4a3c-ab47-dad76c03dc31 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:32.480658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bb2fc-71ab-4a3c-ab47-dad76c03dc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:32.546780 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-928bb2fc-71ab-4a3c-ab47-dad76c03dc31 None None] MFA Rules not processed for user `7aee709fda854364a32f39ff15c6aafa`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:32.548079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-928bb2fc-71ab-4a3c-ab47-dad76c03dc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:32.603838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:32.649753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 659/1318] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:45:32 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:32.691887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44119d-cd59-4e8d-8ed4-76b8da410462 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:32.693471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ad44119d-cd59-4e8d-8ed4-76b8da410462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:32.798059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44119d-cd59-4e8d-8ed4-76b8da410462 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:32.800827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:32.873341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bef1-bb44-4306-8b67-09487ab481ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:32.874088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bef1-bb44-4306-8b67-09487ab481ef service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:32.874290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bef1-bb44-4306-8b67-09487ab481ef service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:32.874563 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bef1-bb44-4306-8b67-09487ab481ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:32.876235 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fa32bef1-bb44-4306-8b67-09487ab481ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:32.985266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bef1-bb44-4306-8b67-09487ab481ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:32.986695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bef1-bb44-4306-8b67-09487ab481ef service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:33.005217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 660/1319] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:32 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:45:35.638607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c076b0e-0282-4b6f-9c37-1e3e41ea886f None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:35.639215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c076b0e-0282-4b6f-9c37-1e3e41ea886f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:35.639215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c076b0e-0282-4b6f-9c37-1e3e41ea886f None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:35.640592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 660/1320] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:45:35.647332 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c5ac0-1484-4094-9026-29ea6efedbe0 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:35.648903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c5ac0-1484-4094-9026-29ea6efedbe0 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:35.648903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c5ac0-1484-4094-9026-29ea6efedbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:35.713372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:35.740047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e50c5ac0-1484-4094-9026-29ea6efedbe0 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:35.741337 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e50c5ac0-1484-4094-9026-29ea6efedbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:35.840681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 661/1321] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:45:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:35.880827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c076b0e-0282-4b6f-9c37-1e3e41ea886f None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:35.882361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9c076b0e-0282-4b6f-9c37-1e3e41ea886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:35.986845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c076b0e-0282-4b6f-9c37-1e3e41ea886f None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:36.062450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b4f46-5532-426c-98a5-54d528b0b437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:36.063097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b4f46-5532-426c-98a5-54d528b0b437 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:36.063381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b4f46-5532-426c-98a5-54d528b0b437 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:36.063655 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b4f46-5532-426c-98a5-54d528b0b437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:36.065346 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6a2b4f46-5532-426c-98a5-54d528b0b437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:36.072197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:36.173708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b4f46-5532-426c-98a5-54d528b0b437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:36.175144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b4f46-5532-426c-98a5-54d528b0b437 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:36.194589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 661/1322] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:45:39.196795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f74b3e-d378-4e82-8777-428d0c52fc28 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:39.197464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f74b3e-d378-4e82-8777-428d0c52fc28 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:39.197464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f74b3e-d378-4e82-8777-428d0c52fc28 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:39.198733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 662/1323] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:39.205205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4628e7-d960-428f-8551-aa6818557d6a None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:39.205952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4628e7-d960-428f-8551-aa6818557d6a None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:39.205952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4628e7-d960-428f-8551-aa6818557d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:39.277163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-2b4628e7-d960-428f-8551-aa6818557d6a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:39.278962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2b4628e7-d960-428f-8551-aa6818557d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:39.344585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:39.387335 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 662/1324] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:39.424875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f74b3e-d378-4e82-8777-428d0c52fc28 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:39.426468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-00f74b3e-d378-4e82-8777-428d0c52fc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:39.538749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f74b3e-d378-4e82-8777-428d0c52fc28 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:39.550912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:39.614531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac5cad-0b26-4e68-b31d-cd5349d64dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:39.615092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac5cad-0b26-4e68-b31d-cd5349d64dcb service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:39.615376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac5cad-0b26-4e68-b31d-cd5349d64dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:39.615655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac5cad-0b26-4e68-b31d-cd5349d64dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:39.617361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a4ac5cad-0b26-4e68-b31d-cd5349d64dcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:39.731539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac5cad-0b26-4e68-b31d-cd5349d64dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:39.732997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac5cad-0b26-4e68-b31d-cd5349d64dcb service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:39.751940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 663/1325] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:45:41.873217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf62a9a-4625-4dfd-978f-d0079676aaf6 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:41.873737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf62a9a-4625-4dfd-978f-d0079676aaf6 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:41.873737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf62a9a-4625-4dfd-978f-d0079676aaf6 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:41.875059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 663/1326] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:41.880677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795bdbe-dad7-47f0-ae19-639366302329 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:41.881151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795bdbe-dad7-47f0-ae19-639366302329 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:41.881270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795bdbe-dad7-47f0-ae19-639366302329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:41.930475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:41.950871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-b795bdbe-dad7-47f0-ae19-639366302329 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:41.952107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b795bdbe-dad7-47f0-ae19-639366302329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:42.068024 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 664/1327] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:42.105089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf62a9a-4625-4dfd-978f-d0079676aaf6 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:42.107398 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2bf62a9a-4625-4dfd-978f-d0079676aaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:42.225123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf62a9a-4625-4dfd-978f-d0079676aaf6 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:42.319142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ab91c-dd9a-40de-a6f0-a4c4e621e86b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:42.319367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ab91c-dd9a-40de-a6f0-a4c4e621e86b service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:42.319946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ab91c-dd9a-40de-a6f0-a4c4e621e86b service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:42.319946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ab91c-dd9a-40de-a6f0-a4c4e621e86b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:42.321575 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3c4ab91c-dd9a-40de-a6f0-a4c4e621e86b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:42.340476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:42.455433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ab91c-dd9a-40de-a6f0-a4c4e621e86b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:42.457076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ab91c-dd9a-40de-a6f0-a4c4e621e86b service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:42.478162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 664/1328] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:45:44.642534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e227a86-eed3-4364-b7f6-886290c09eb4 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:44.643298 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e227a86-eed3-4364-b7f6-886290c09eb4 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:44.643298 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e227a86-eed3-4364-b7f6-886290c09eb4 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:44.644664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 665/1329] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:44.651487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d8d36-b447-4222-ab06-3b5215475cc6 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:44.652021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d8d36-b447-4222-ab06-3b5215475cc6 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:44.652131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d8d36-b447-4222-ab06-3b5215475cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:44.731455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-8b8d8d36-b447-4222-ab06-3b5215475cc6 None None] MFA Rules not processed for user `7aee709fda854364a32f39ff15c6aafa`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:44.732745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8b8d8d36-b447-4222-ab06-3b5215475cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:44.813623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:44.850006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 665/1330] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:45:44 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:44.888295 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e227a86-eed3-4364-b7f6-886290c09eb4 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:44.889940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3e227a86-eed3-4364-b7f6-886290c09eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:45.003644 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e227a86-eed3-4364-b7f6-886290c09eb4 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:45.029802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:45.089234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc092458-a345-41bb-a988-f4246999dbcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:45.089889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc092458-a345-41bb-a988-f4246999dbcc service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:45.090170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc092458-a345-41bb-a988-f4246999dbcc service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:45.090443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc092458-a345-41bb-a988-f4246999dbcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:45.092796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dc092458-a345-41bb-a988-f4246999dbcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:45.234314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc092458-a345-41bb-a988-f4246999dbcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:45.235784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc092458-a345-41bb-a988-f4246999dbcc service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:45.256718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 666/1331] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:45:47.671119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa9d30-16f0-40b4-8b19-dee24294cfeb None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:47.671714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa9d30-16f0-40b4-8b19-dee24294cfeb None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:47.671714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa9d30-16f0-40b4-8b19-dee24294cfeb None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:47.673072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 666/1332] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:47.678785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779158-ed13-4bfa-b2ed-cfa57fe772b5 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:47.679265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779158-ed13-4bfa-b2ed-cfa57fe772b5 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:47.679381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30779158-ed13-4bfa-b2ed-cfa57fe772b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:47.745650 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:47.750949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-30779158-ed13-4bfa-b2ed-cfa57fe772b5 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:47.752607 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-30779158-ed13-4bfa-b2ed-cfa57fe772b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:47.857197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 667/1333] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:47.893516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa9d30-16f0-40b4-8b19-dee24294cfeb None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:47.895080 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f7fa9d30-16f0-40b4-8b19-dee24294cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:48.004860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa9d30-16f0-40b4-8b19-dee24294cfeb None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:48.079298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84091b-7aa5-48dd-be7d-f032c7c4eb47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:48.079947 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84091b-7aa5-48dd-be7d-f032c7c4eb47 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:48.080233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84091b-7aa5-48dd-be7d-f032c7c4eb47 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:48.080550 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84091b-7aa5-48dd-be7d-f032c7c4eb47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:48.082236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-cb84091b-7aa5-48dd-be7d-f032c7c4eb47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:48.111435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:48.194319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84091b-7aa5-48dd-be7d-f032c7c4eb47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:48.195753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84091b-7aa5-48dd-be7d-f032c7c4eb47 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:48.217007 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 667/1334] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:45:50.372644 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbe7ee-4276-4366-bcd5-67dc1fb626b0 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:50.373207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbe7ee-4276-4366-bcd5-67dc1fb626b0 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:50.373207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbe7ee-4276-4366-bcd5-67dc1fb626b0 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:50.374564 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 668/1335] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:50.379974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296c605-ecd9-4d08-8c15-ea300b792bc2 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:50.380474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296c605-ecd9-4d08-8c15-ea300b792bc2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:50.380570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296c605-ecd9-4d08-8c15-ea300b792bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:50.451801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-d296c605-ecd9-4d08-8c15-ea300b792bc2 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:50.453124 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d296c605-ecd9-4d08-8c15-ea300b792bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:50.542138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:50.568422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 668/1336] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:50.609381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbe7ee-4276-4366-bcd5-67dc1fb626b0 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:50.610974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c2dbe7ee-4276-4366-bcd5-67dc1fb626b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:50.736508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbe7ee-4276-4366-bcd5-67dc1fb626b0 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:50.774849 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:50.832134 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f7cab-62e2-4063-9741-5546550bcc8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:50.832742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f7cab-62e2-4063-9741-5546550bcc8c service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:50.833027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f7cab-62e2-4063-9741-5546550bcc8c service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:50.833301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f7cab-62e2-4063-9741-5546550bcc8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:50.835086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-897f7cab-62e2-4063-9741-5546550bcc8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:50.991389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f7cab-62e2-4063-9741-5546550bcc8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:50.993395 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f7cab-62e2-4063-9741-5546550bcc8c service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:51.015034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 669/1337] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:45:53.807048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5b38a-2de3-4ed1-a9f1-f89d4e6d5e55 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:53.807666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5b38a-2de3-4ed1-a9f1-f89d4e6d5e55 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:53.807891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5b38a-2de3-4ed1-a9f1-f89d4e6d5e55 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:53.810696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 669/1338] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:45:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:53.820099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c049010-569f-49db-aaa8-a654814b5f2e None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:53.820769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c049010-569f-49db-aaa8-a654814b5f2e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:53.821157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c049010-569f-49db-aaa8-a654814b5f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:53.936577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-6c049010-569f-49db-aaa8-a654814b5f2e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:53.938590 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6c049010-569f-49db-aaa8-a654814b5f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:53.954293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:54.088072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 670/1339] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:45:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:45:54.158052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5b38a-2de3-4ed1-a9f1-f89d4e6d5e55 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:54.159848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-37d5b38a-2de3-4ed1-a9f1-f89d4e6d5e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:54.287907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5b38a-2de3-4ed1-a9f1-f89d4e6d5e55 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:45:54.397950 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44dc2f-c8cf-4b14-982e-8c60ad137c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:45:54.398585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44dc2f-c8cf-4b14-982e-8c60ad137c09 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:54.398916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44dc2f-c8cf-4b14-982e-8c60ad137c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:54.399240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44dc2f-c8cf-4b14-982e-8c60ad137c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:54.401121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1c44dc2f-c8cf-4b14-982e-8c60ad137c09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:54.447582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:54.566945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44dc2f-c8cf-4b14-982e-8c60ad137c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:45:54.569573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44dc2f-c8cf-4b14-982e-8c60ad137c09 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:45:54.605783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 670/1340] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:45:59.570908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61e6d-4fc7-43c0-af9d-ff0a752d19a1 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:59.571805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61e6d-4fc7-43c0-af9d-ff0a752d19a1 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:59.571805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a61e6d-4fc7-43c0-af9d-ff0a752d19a1 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:59.573118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 671/1341] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:45:59.579099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302d607-05fe-4b58-be13-fcd3a2230500 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:45:59.579894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302d607-05fe-4b58-be13-fcd3a2230500 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:45:59.579894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302d607-05fe-4b58-be13-fcd3a2230500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:45:59.673749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0302d607-05fe-4b58-be13-fcd3a2230500 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:45:59.675328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0302d607-05fe-4b58-be13-fcd3a2230500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:45:59.805698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:45:59.824372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 671/1342] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:45:59.864182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61e6d-4fc7-43c0-af9d-ff0a752d19a1 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:45:59.865936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b1a61e6d-4fc7-43c0-af9d-ff0a752d19a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:00.013436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a61e6d-4fc7-43c0-af9d-ff0a752d19a1 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:00.074571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:00.115605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0835c-5b0b-4780-bbd7-ca20d5baa2d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:00.116262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0835c-5b0b-4780-bbd7-ca20d5baa2d1 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:00.116574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0835c-5b0b-4780-bbd7-ca20d5baa2d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:00.116849 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0835c-5b0b-4780-bbd7-ca20d5baa2d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:00.118523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-44a0835c-5b0b-4780-bbd7-ca20d5baa2d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:00.291496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0835c-5b0b-4780-bbd7-ca20d5baa2d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:00.292893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0835c-5b0b-4780-bbd7-ca20d5baa2d1 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:00.315506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 672/1343] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:45:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:03.575580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655349b2-bf61-43eb-a454-9aab5defd9fd None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:03.576188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655349b2-bf61-43eb-a454-9aab5defd9fd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:03.576377 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655349b2-bf61-43eb-a454-9aab5defd9fd None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:03.578381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 672/1344] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:03.585650 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47724ee-3018-4ddd-8128-7b662d637bda None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:03.586406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47724ee-3018-4ddd-8128-7b662d637bda None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:03.586708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47724ee-3018-4ddd-8128-7b662d637bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:03.664405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d47724ee-3018-4ddd-8128-7b662d637bda None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:03.665734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d47724ee-3018-4ddd-8128-7b662d637bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:03.690359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:03.783641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 673/1345] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:03.825817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655349b2-bf61-43eb-a454-9aab5defd9fd None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:03.827939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-655349b2-bf61-43eb-a454-9aab5defd9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:03.997648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655349b2-bf61-43eb-a454-9aab5defd9fd None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:04.108921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0472a-4b13-41c5-9bf6-bd2077005df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:04.109663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0472a-4b13-41c5-9bf6-bd2077005df7 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:04.109952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0472a-4b13-41c5-9bf6-bd2077005df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:04.110232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0472a-4b13-41c5-9bf6-bd2077005df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:04.111966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f0f0472a-4b13-41c5-9bf6-bd2077005df7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:04.179547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:04.322927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0472a-4b13-41c5-9bf6-bd2077005df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:04.324567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0472a-4b13-41c5-9bf6-bd2077005df7 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:04.350027 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 673/1346] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:09.759328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490082a2-8291-4321-b296-98c6b7dce5bd None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:09.760028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490082a2-8291-4321-b296-98c6b7dce5bd None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:09.760157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490082a2-8291-4321-b296-98c6b7dce5bd None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:09.762686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 674/1347] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:09.772096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d456677-73f8-46f9-99cd-adcd41c7729d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:09.772971 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d456677-73f8-46f9-99cd-adcd41c7729d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:09.773379 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d456677-73f8-46f9-99cd-adcd41c7729d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:09.900086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-7d456677-73f8-46f9-99cd-adcd41c7729d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:09.902258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7d456677-73f8-46f9-99cd-adcd41c7729d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:10.058189 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:10.069822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 674/1348] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:10.118657 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490082a2-8291-4321-b296-98c6b7dce5bd None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:10.120196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-490082a2-8291-4321-b296-98c6b7dce5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:10.309693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490082a2-8291-4321-b296-98c6b7dce5bd None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:10.384808 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:10.416827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd44396-05ac-4cc4-a71e-3b068126e381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:10.417503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd44396-05ac-4cc4-a71e-3b068126e381 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:10.417804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd44396-05ac-4cc4-a71e-3b068126e381 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:10.418059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd44396-05ac-4cc4-a71e-3b068126e381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:10.419844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4bd44396-05ac-4cc4-a71e-3b068126e381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:10.562560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd44396-05ac-4cc4-a71e-3b068126e381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:10.563350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd44396-05ac-4cc4-a71e-3b068126e381 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:10.583839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 675/1349] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:15.776167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f01f40-67cd-4f3b-91f9-3ccee2778a74 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:15.776816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f01f40-67cd-4f3b-91f9-3ccee2778a74 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:15.778279 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f01f40-67cd-4f3b-91f9-3ccee2778a74 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:15.780555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 675/1350] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 20:46:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:15.786520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335bc12-c112-43a8-9d1e-8efe58600fea None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:15.787463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335bc12-c112-43a8-9d1e-8efe58600fea None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:15.787949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335bc12-c112-43a8-9d1e-8efe58600fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:15.896586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-8335bc12-c112-43a8-9d1e-8efe58600fea None None] MFA Rules not processed for user `fffebfc51dba47f68263c0bcd6b18d22`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:15.897819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8335bc12-c112-43a8-9d1e-8efe58600fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:15.943651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:16.130639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 676/1351] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 20:46:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:46:20.367083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb1f84-9464-48b3-b6ed-c875c38e7cbf None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:20.367622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb1f84-9464-48b3-b6ed-c875c38e7cbf None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:20.367622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb1f84-9464-48b3-b6ed-c875c38e7cbf None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:20.369296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 676/1352] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 20:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:20.374909 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd0171-cdae-4ce4-a411-834c2eea8701 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:20.375410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd0171-cdae-4ce4-a411-834c2eea8701 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:20.375517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd0171-cdae-4ce4-a411-834c2eea8701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:20.459707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-ecbd0171-cdae-4ce4-a411-834c2eea8701 None None] MFA Rules not processed for user `fffebfc51dba47f68263c0bcd6b18d22`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:20.460957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ecbd0171-cdae-4ce4-a411-834c2eea8701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:20.557601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:20.599848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 677/1353] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 20:46:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:22.165483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31751e-1098-4680-9ac1-6b6552543085 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:22.166009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31751e-1098-4680-9ac1-6b6552543085 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:22.166169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31751e-1098-4680-9ac1-6b6552543085 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:22.167878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 677/1354] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 20:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:22.173630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a7e91-0fc8-4d8d-b4a0-fb09f21a460d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:22.174366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a7e91-0fc8-4d8d-b4a0-fb09f21a460d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:22.174366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a7e91-0fc8-4d8d-b4a0-fb09f21a460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:22.258897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-dd3a7e91-0fc8-4d8d-b4a0-fb09f21a460d None None] MFA Rules not processed for user `fffebfc51dba47f68263c0bcd6b18d22`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:22.260115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dd3a7e91-0fc8-4d8d-b4a0-fb09f21a460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:22.375685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 678/1355] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 20:46:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:46:25.922665 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ee830-0e27-4aab-b5ba-5a66ee307420 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:25.923526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ee830-0e27-4aab-b5ba-5a66ee307420 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:25.923526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ee830-0e27-4aab-b5ba-5a66ee307420 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:25.927951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 678/1356] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:25.931784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c2b9-3ff1-4860-89f4-9d5f52efccd1 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:25.932302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c2b9-3ff1-4860-89f4-9d5f52efccd1 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:25.932439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c2b9-3ff1-4860-89f4-9d5f52efccd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:25.986877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:26.036799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-b531c2b9-3ff1-4860-89f4-9d5f52efccd1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:26.038609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b531c2b9-3ff1-4860-89f4-9d5f52efccd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:26.236081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 679/1357] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:26.309987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dd81d-e83d-4c09-9d0d-1ac90664e0f2 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:26.310439 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dd81d-e83d-4c09-9d0d-1ac90664e0f2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:26.310564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dd81d-e83d-4c09-9d0d-1ac90664e0f2 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:26.311914 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 679/1358] 10.222.0.85 () {54 vars in 962 bytes} [Tue Jul 19 20:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:26.319878 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29d15b-87fe-442f-8b1b-12b3f6e2bdc5 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:26.320642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29d15b-87fe-442f-8b1b-12b3f6e2bdc5 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:26.321200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29d15b-87fe-442f-8b1b-12b3f6e2bdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:26.431050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-8b29d15b-87fe-442f-8b1b-12b3f6e2bdc5 None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:26.434260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8b29d15b-87fe-442f-8b1b-12b3f6e2bdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:26.515773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:26.596118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 680/1359] 10.222.0.85 () {58 vars in 1059 bytes} [Tue Jul 19 20:46:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:26.604840 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dd81d-e83d-4c09-9d0d-1ac90664e0f2 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:26.607214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f82dd81d-e83d-4c09-9d0d-1ac90664e0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:26.862459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dd81d-e83d-4c09-9d0d-1ac90664e0f2 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:26.961041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:27.007927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea718d26-95c3-44f9-bb20-6ff76f5aeea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:27.008817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea718d26-95c3-44f9-bb20-6ff76f5aeea3 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:27.009205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea718d26-95c3-44f9-bb20-6ff76f5aeea3 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:27.009581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea718d26-95c3-44f9-bb20-6ff76f5aeea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:27.011347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ea718d26-95c3-44f9-bb20-6ff76f5aeea3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:27.107353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea718d26-95c3-44f9-bb20-6ff76f5aeea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:27.109311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea718d26-95c3-44f9-bb20-6ff76f5aeea3 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:27.131373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 680/1360] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:29.572466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fd364-5aeb-4b01-a982-bc460e841c97 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:29.573098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fd364-5aeb-4b01-a982-bc460e841c97 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:29.573098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fd364-5aeb-4b01-a982-bc460e841c97 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:29.575111 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 681/1361] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:29.580455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07369-788d-4fa5-a14b-871cf20343e3 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:29.580928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07369-788d-4fa5-a14b-871cf20343e3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:29.581023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07369-788d-4fa5-a14b-871cf20343e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:29.659646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c2b07369-788d-4fa5-a14b-871cf20343e3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:29.661477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c2b07369-788d-4fa5-a14b-871cf20343e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:29.713900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:29.785410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 681/1362] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:29.842810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fd364-5aeb-4b01-a982-bc460e841c97 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:29.844233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-251fd364-5aeb-4b01-a982-bc460e841c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:30.083570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fd364-5aeb-4b01-a982-bc460e841c97 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:30.094139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:30.239658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbe04c-7d3d-4673-bc4d-adf300a76bdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:30.240372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbe04c-7d3d-4673-bc4d-adf300a76bdd service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:30.240670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbe04c-7d3d-4673-bc4d-adf300a76bdd service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:30.240947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbe04c-7d3d-4673-bc4d-adf300a76bdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:30.243055 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4cbbe04c-7d3d-4673-bc4d-adf300a76bdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:30.345024 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbe04c-7d3d-4673-bc4d-adf300a76bdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:30.346864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbe04c-7d3d-4673-bc4d-adf300a76bdd service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:30.368888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 682/1363] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:32.836903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c287083-2612-402e-b08e-cf5562a42b3f None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:32.837487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c287083-2612-402e-b08e-cf5562a42b3f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:32.837487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c287083-2612-402e-b08e-cf5562a42b3f None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:32.843436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 682/1364] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:32.845264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72acda-4249-4b21-9687-24f4752dc62e None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:32.845879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72acda-4249-4b21-9687-24f4752dc62e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:32.845879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72acda-4249-4b21-9687-24f4752dc62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:32.862270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:32.919467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-6e72acda-4249-4b21-9687-24f4752dc62e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:32.920903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6e72acda-4249-4b21-9687-24f4752dc62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:33.031493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 683/1365] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:33.086967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa8fd5-9630-4b95-ad24-d2dc1c2c4ba2 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:33.087372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa8fd5-9630-4b95-ad24-d2dc1c2c4ba2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:33.087726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa8fd5-9630-4b95-ad24-d2dc1c2c4ba2 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:33.089663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 683/1366] 10.222.0.85 () {54 vars in 962 bytes} [Tue Jul 19 20:46:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:33.095501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0694898-49ef-4709-88dd-776b7663be79 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:33.096099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0694898-49ef-4709-88dd-776b7663be79 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:33.096789 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0694898-49ef-4709-88dd-776b7663be79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:33.179369 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c0694898-49ef-4709-88dd-776b7663be79 None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:33.180601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c0694898-49ef-4709-88dd-776b7663be79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:33.230151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:33.327195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 684/1367] 10.222.0.85 () {58 vars in 1059 bytes} [Tue Jul 19 20:46:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:33.335399 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa8fd5-9630-4b95-ad24-d2dc1c2c4ba2 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:33.337352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4dfa8fd5-9630-4b95-ad24-d2dc1c2c4ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:33.499366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa8fd5-9630-4b95-ad24-d2dc1c2c4ba2 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:33.521072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:33.630351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3653384-d8f0-471e-95d6-7f6289ae59b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:33.631002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3653384-d8f0-471e-95d6-7f6289ae59b1 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:33.631284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3653384-d8f0-471e-95d6-7f6289ae59b1 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:33.631557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3653384-d8f0-471e-95d6-7f6289ae59b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:33.633438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a3653384-d8f0-471e-95d6-7f6289ae59b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:33.740683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3653384-d8f0-471e-95d6-7f6289ae59b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:33.743253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3653384-d8f0-471e-95d6-7f6289ae59b1 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:33.773341 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 684/1368] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:34.704476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db22b-2625-49e1-ba25-1dbbff4f67a7 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:34.705822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db22b-2625-49e1-ba25-1dbbff4f67a7 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:34.706232 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db22b-2625-49e1-ba25-1dbbff4f67a7 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:34.709405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 685/1369] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:34.715583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354d042-a6b8-476b-bb41-aa565b2be423 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:34.716103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354d042-a6b8-476b-bb41-aa565b2be423 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:34.716373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354d042-a6b8-476b-bb41-aa565b2be423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:34.751288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:34.798088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-2354d042-a6b8-476b-bb41-aa565b2be423 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:34.800980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2354d042-a6b8-476b-bb41-aa565b2be423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:34.946784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 685/1370] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:35.274211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae41f3-b986-407b-ace9-3a413ad1ddfc None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:35.274505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae41f3-b986-407b-ace9-3a413ad1ddfc None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:35.274760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae41f3-b986-407b-ace9-3a413ad1ddfc None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:35.276414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 686/1371] 10.222.0.85 () {54 vars in 964 bytes} [Tue Jul 19 20:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:46:35.282875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8123f-deaf-467f-adbc-047097dbf9b9 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:35.283325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8123f-deaf-467f-adbc-047097dbf9b9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:35.283819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8123f-deaf-467f-adbc-047097dbf9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:35.402950 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-bba8123f-deaf-467f-adbc-047097dbf9b9 None None] MFA Rules not processed for user `81411204bc9f4210880d807b41da9643`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:35.404785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-bba8123f-deaf-467f-adbc-047097dbf9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:35.496673 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:35.546805 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 686/1372] 10.222.0.85 () {58 vars in 1061 bytes} [Tue Jul 19 20:46:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:35.556302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae41f3-b986-407b-ace9-3a413ad1ddfc None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:35.558025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-34ae41f3-b986-407b-ace9-3a413ad1ddfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:35.694287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae41f3-b986-407b-ace9-3a413ad1ddfc None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:35.747706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:35.819509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326f3d3-3c11-44f7-8845-ab3f609a5de9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:35.820816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326f3d3-3c11-44f7-8845-ab3f609a5de9 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:35.821500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326f3d3-3c11-44f7-8845-ab3f609a5de9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:35.822051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326f3d3-3c11-44f7-8845-ab3f609a5de9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:35.826177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c326f3d3-3c11-44f7-8845-ab3f609a5de9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:36.108690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326f3d3-3c11-44f7-8845-ab3f609a5de9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:36.111068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326f3d3-3c11-44f7-8845-ab3f609a5de9 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:36.138957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 687/1373] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:36.171823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0f4b-60d3-45e0-8570-58a9e3ae52f9 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:36.172447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0f4b-60d3-45e0-8570-58a9e3ae52f9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:36.172447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0f4b-60d3-45e0-8570-58a9e3ae52f9 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:36.173894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 687/1374] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:46:36.180869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a177dd-51c9-42bc-becc-14805c05a701 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:36.181261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a177dd-51c9-42bc-becc-14805c05a701 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:36.181546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a177dd-51c9-42bc-becc-14805c05a701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:36.263100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d9a177dd-51c9-42bc-becc-14805c05a701 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:36.264616 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d9a177dd-51c9-42bc-becc-14805c05a701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:36.291363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:36.404489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 688/1375] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:36.453329 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada0f4b-60d3-45e0-8570-58a9e3ae52f9 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:36.455690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4ada0f4b-60d3-45e0-8570-58a9e3ae52f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:36.636479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada0f4b-60d3-45e0-8570-58a9e3ae52f9 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:36.705016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:36.785957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abbaa9-21b0-4c26-b946-3989c737eb47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:36.786609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abbaa9-21b0-4c26-b946-3989c737eb47 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:36.786889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abbaa9-21b0-4c26-b946-3989c737eb47 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:36.787160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abbaa9-21b0-4c26-b946-3989c737eb47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:36.788860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-25abbaa9-21b0-4c26-b946-3989c737eb47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:36.909845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abbaa9-21b0-4c26-b946-3989c737eb47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:36.911810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abbaa9-21b0-4c26-b946-3989c737eb47 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:36.941530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 688/1376] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:39.126684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288438f7-cbc4-4183-a431-caa33ffe96c3 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:39.127478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288438f7-cbc4-4183-a431-caa33ffe96c3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:39.127478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288438f7-cbc4-4183-a431-caa33ffe96c3 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:39.136255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 689/1377] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:46:39.139992 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e419f-eedb-4f95-8047-b7c0a91fffbd None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:39.140570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e419f-eedb-4f95-8047-b7c0a91fffbd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:39.141045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e419f-eedb-4f95-8047-b7c0a91fffbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:39.342935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c67e419f-eedb-4f95-8047-b7c0a91fffbd None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:39.344296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c67e419f-eedb-4f95-8047-b7c0a91fffbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:39.351307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:39.478468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 689/1378] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:39.524283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288438f7-cbc4-4183-a431-caa33ffe96c3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:39.525899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-288438f7-cbc4-4183-a431-caa33ffe96c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:39.678787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288438f7-cbc4-4183-a431-caa33ffe96c3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:39.682405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:39.813451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0289ff46-ffc0-4980-aa25-5657da474630 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:39.814126 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289ff46-ffc0-4980-aa25-5657da474630 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:39.814409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289ff46-ffc0-4980-aa25-5657da474630 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:39.814687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289ff46-ffc0-4980-aa25-5657da474630 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:39.816637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0289ff46-ffc0-4980-aa25-5657da474630 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:39.930959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0289ff46-ffc0-4980-aa25-5657da474630 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:39.932700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0289ff46-ffc0-4980-aa25-5657da474630 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:39.957111 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 690/1379] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:42.120975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997e58c-ff1b-44bd-acd1-34a05ddbd8a3 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:42.121703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997e58c-ff1b-44bd-acd1-34a05ddbd8a3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:42.121703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997e58c-ff1b-44bd-acd1-34a05ddbd8a3 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:42.123224 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 690/1380] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:42.128870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d68fcb-02b0-4269-8b93-d4f557751a71 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:42.129402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d68fcb-02b0-4269-8b93-d4f557751a71 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:42.129402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d68fcb-02b0-4269-8b93-d4f557751a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:42.140642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:42.207498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d7d68fcb-02b0-4269-8b93-d4f557751a71 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:42.209145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d7d68fcb-02b0-4269-8b93-d4f557751a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:42.317215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 691/1381] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:42.363777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997e58c-ff1b-44bd-acd1-34a05ddbd8a3 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:42.365873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e997e58c-ff1b-44bd-acd1-34a05ddbd8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:42.559507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:42.571942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997e58c-ff1b-44bd-acd1-34a05ddbd8a3 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:42.703771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69fab1-3121-4ccc-9bf5-93f8a965aa23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:42.704465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69fab1-3121-4ccc-9bf5-93f8a965aa23 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:42.704756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69fab1-3121-4ccc-9bf5-93f8a965aa23 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:42.705035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69fab1-3121-4ccc-9bf5-93f8a965aa23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:42.706744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4f69fab1-3121-4ccc-9bf5-93f8a965aa23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:42.809498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69fab1-3121-4ccc-9bf5-93f8a965aa23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:42.811831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69fab1-3121-4ccc-9bf5-93f8a965aa23 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:42.824847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:42.835797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 691/1382] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:45.099807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81e30c-d394-4b35-bb5a-7719bcb8c610 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:45.100435 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81e30c-d394-4b35-bb5a-7719bcb8c610 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:45.100435 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81e30c-d394-4b35-bb5a-7719bcb8c610 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:45.101810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 692/1383] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:45.108076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf5a3e-28fb-4abb-9cb8-cef6af83fd3d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:45.108666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf5a3e-28fb-4abb-9cb8-cef6af83fd3d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:45.108666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf5a3e-28fb-4abb-9cb8-cef6af83fd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:45.179314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-2bbf5a3e-28fb-4abb-9cb8-cef6af83fd3d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:45.180580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2bbf5a3e-28fb-4abb-9cb8-cef6af83fd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:45.302775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 692/1384] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:45.345038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81e30c-d394-4b35-bb5a-7719bcb8c610 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:45.346592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2e81e30c-d394-4b35-bb5a-7719bcb8c610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:45.442682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:45.506788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81e30c-d394-4b35-bb5a-7719bcb8c610 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:45.631513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602cfa3-5d1b-4ccd-90ef-4620eb8f3062 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:45.632421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602cfa3-5d1b-4ccd-90ef-4620eb8f3062 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:45.632819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602cfa3-5d1b-4ccd-90ef-4620eb8f3062 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:45.633176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602cfa3-5d1b-4ccd-90ef-4620eb8f3062 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:45.635412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2602cfa3-5d1b-4ccd-90ef-4620eb8f3062 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:45.696199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:45.742975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602cfa3-5d1b-4ccd-90ef-4620eb8f3062 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:45.744787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602cfa3-5d1b-4ccd-90ef-4620eb8f3062 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:45.763729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 693/1385] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:48.020280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e6d16-bf56-4533-9722-1193b2933aab None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:48.020906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e6d16-bf56-4533-9722-1193b2933aab None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:48.020906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e6d16-bf56-4533-9722-1193b2933aab None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:48.022243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 693/1386] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:48.028436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d95fb-e450-4349-91eb-30ebebcfe707 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:48.028979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d95fb-e450-4349-91eb-30ebebcfe707 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:48.028979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d95fb-e450-4349-91eb-30ebebcfe707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:48.115035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-915d95fb-e450-4349-91eb-30ebebcfe707 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:48.116254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-915d95fb-e450-4349-91eb-30ebebcfe707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:48.249584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 694/1387] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:48.305277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e6d16-bf56-4533-9722-1193b2933aab None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:48.306826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-156e6d16-bf56-4533-9722-1193b2933aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:48.399342 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:48.481434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e6d16-bf56-4533-9722-1193b2933aab None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:48.614369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9637f-928b-4bbb-bda1-457e48b4fdeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:48.614923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9637f-928b-4bbb-bda1-457e48b4fdeb service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:48.615207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9637f-928b-4bbb-bda1-457e48b4fdeb service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:48.615480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9637f-928b-4bbb-bda1-457e48b4fdeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:48.617233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-eed9637f-928b-4bbb-bda1-457e48b4fdeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:48.659256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:48.718357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9637f-928b-4bbb-bda1-457e48b4fdeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:48.720464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9637f-928b-4bbb-bda1-457e48b4fdeb service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:48.740097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 694/1388] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:51.392724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48317d-a7ff-451c-8bc2-6e36961c9d4f None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:51.393425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48317d-a7ff-451c-8bc2-6e36961c9d4f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:51.393425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48317d-a7ff-451c-8bc2-6e36961c9d4f None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:51.394699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 695/1389] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:51.400418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a90df6-d158-4ab9-a0ac-ef28081ad0e7 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:51.400954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a90df6-d158-4ab9-a0ac-ef28081ad0e7 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:51.400954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a90df6-d158-4ab9-a0ac-ef28081ad0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:51.471001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-a0a90df6-d158-4ab9-a0ac-ef28081ad0e7 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:51.472234 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a0a90df6-d158-4ab9-a0ac-ef28081ad0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:51.575587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 695/1390] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:51.620252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48317d-a7ff-451c-8bc2-6e36961c9d4f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:51.622398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7a48317d-a7ff-451c-8bc2-6e36961c9d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:51.654482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:51.802518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48317d-a7ff-451c-8bc2-6e36961c9d4f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:51.919835 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:51.932306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94679070-206d-4e5a-84e0-6d97ae673880 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:51.932991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679070-206d-4e5a-84e0-6d97ae673880 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:51.933347 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679070-206d-4e5a-84e0-6d97ae673880 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:51.934379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679070-206d-4e5a-84e0-6d97ae673880 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:51.936057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-94679070-206d-4e5a-84e0-6d97ae673880 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:51.981182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f744d4-db2f-402b-aaad-761063f5c7dd None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:51.981471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f744d4-db2f-402b-aaad-761063f5c7dd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:51.981752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f744d4-db2f-402b-aaad-761063f5c7dd None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:51.983206 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 696/1391] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:51.989186 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fea93-afe5-49df-b6a5-c2f2c154b1c4 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:51.989478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fea93-afe5-49df-b6a5-c2f2c154b1c4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:51.989761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fea93-afe5-49df-b6a5-c2f2c154b1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:52.005904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:52.037629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94679070-206d-4e5a-84e0-6d97ae673880 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:52.039306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94679070-206d-4e5a-84e0-6d97ae673880 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:52.063010 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 696/1392] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:52.064627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e58fea93-afe5-49df-b6a5-c2f2c154b1c4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:52.066168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e58fea93-afe5-49df-b6a5-c2f2c154b1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:52.211539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 697/1393] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:46:52.271527 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd162d75-b9da-481d-bb2e-0ebdabd72305 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:52.271959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd162d75-b9da-481d-bb2e-0ebdabd72305 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:52.272356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd162d75-b9da-481d-bb2e-0ebdabd72305 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:52.273850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 697/1394] 10.222.0.85 () {54 vars in 966 bytes} [Tue Jul 19 20:46:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:52.279994 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7a2b6-150d-4962-bed1-06da8d89e6fe None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:52.280291 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7a2b6-150d-4962-bed1-06da8d89e6fe None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:52.280593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7a2b6-150d-4962-bed1-06da8d89e6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:52.360108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-46f7a2b6-150d-4962-bed1-06da8d89e6fe None None] MFA Rules not processed for user `e35e7720e3bf4c9e80bdefa6a1cb8296`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:52.361271 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-46f7a2b6-150d-4962-bed1-06da8d89e6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:52.399395 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:52.449820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 698/1395] 10.222.0.85 () {58 vars in 1063 bytes} [Tue Jul 19 20:46:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:52.460087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd162d75-b9da-481d-bb2e-0ebdabd72305 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:52.462533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cd162d75-b9da-481d-bb2e-0ebdabd72305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:52.575623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd162d75-b9da-481d-bb2e-0ebdabd72305 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:52.590051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:52.658937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d66edb-a112-4a4a-aba1-72cbd537e7c6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35e7720e3bf4c9e80bdefa6a1cb8296', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:52.659500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66edb-a112-4a4a-aba1-72cbd537e7c6 service glance] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:52.659787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66edb-a112-4a4a-aba1-72cbd537e7c6 service glance] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:52.660061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66edb-a112-4a4a-aba1-72cbd537e7c6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:52.661785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-95d66edb-a112-4a4a-aba1-72cbd537e7c6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:52.789061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d66edb-a112-4a4a-aba1-72cbd537e7c6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:52.790501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d66edb-a112-4a4a-aba1-72cbd537e7c6 service glance] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:52.812136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 698/1396] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:46:53.536495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034b3aa-e430-40ad-887d-905be5002821 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:53.536986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034b3aa-e430-40ad-887d-905be5002821 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:53.537093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034b3aa-e430-40ad-887d-905be5002821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:53.606556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-a034b3aa-e430-40ad-887d-905be5002821 None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:53.607809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a034b3aa-e430-40ad-887d-905be5002821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:53.725832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 699/1397] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 20:46:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:53.776568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4ad4-10ec-4f6f-977c-dc474e30f5a9 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:53.777101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4ad4-10ec-4f6f-977c-dc474e30f5a9 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:53.777101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4ad4-10ec-4f6f-977c-dc474e30f5a9 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:53.778488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 699/1398] 10.222.0.85 () {54 vars in 943 bytes} [Tue Jul 19 20:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:53.785094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69fc27-628b-4e20-bcb6-1b29a6431a2c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:53.785388 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69fc27-628b-4e20-bcb6-1b29a6431a2c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:53.785669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69fc27-628b-4e20-bcb6-1b29a6431a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:53.843794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:53.864821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-1e69fc27-628b-4e20-bcb6-1b29a6431a2c None None] MFA Rules not processed for user `812342d822f54aaf98e324d5d4e56753`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:53.866055 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1e69fc27-628b-4e20-bcb6-1b29a6431a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:53.979532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 700/1399] 10.222.0.85 () {58 vars in 1040 bytes} [Tue Jul 19 20:46:53 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:46:53.990583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d4ad4-10ec-4f6f-977c-dc474e30f5a9 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:53.992643 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-009d4ad4-10ec-4f6f-977c-dc474e30f5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:54.039582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:54.139044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d4ad4-10ec-4f6f-977c-dc474e30f5a9 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:54.219365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f053f06-5863-4edc-83be-fa1cf370e812 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:54.219660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f053f06-5863-4edc-83be-fa1cf370e812 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:54.219941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f053f06-5863-4edc-83be-fa1cf370e812 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:54.221423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 701/1400] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:46:54.227143 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dca133-be76-4b01-9159-afb1eb5baf10 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:54.227675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dca133-be76-4b01-9159-afb1eb5baf10 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:54.227952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dca133-be76-4b01-9159-afb1eb5baf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:54.250346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb98ae-10ce-4280-a785-3120e148759a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:54.251001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb98ae-10ce-4280-a785-3120e148759a service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:54.251279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb98ae-10ce-4280-a785-3120e148759a service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:54.251552 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb98ae-10ce-4280-a785-3120e148759a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:54.253333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-49fb98ae-10ce-4280-a785-3120e148759a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:54.286335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:54.295024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-74dca133-be76-4b01-9159-afb1eb5baf10 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:54.296171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-74dca133-be76-4b01-9159-afb1eb5baf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:54.343600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb98ae-10ce-4280-a785-3120e148759a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:54.345354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb98ae-10ce-4280-a785-3120e148759a service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:54.354365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 700/1401] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 20:46:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:46:54.355690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:54.401609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 702/1402] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:46:54.450725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb98ae-10ce-4280-a785-3120e148759a service swift] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:54.450725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-49fb98ae-10ce-4280-a785-3120e148759a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:54.569562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb98ae-10ce-4280-a785-3120e148759a service swift] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:54.663977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:54.715421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd221d1-e5cf-4b40-8049-1d4ae38a7d1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:54.716116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd221d1-e5cf-4b40-8049-1d4ae38a7d1d service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:54.716466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd221d1-e5cf-4b40-8049-1d4ae38a7d1d service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:54.716778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd221d1-e5cf-4b40-8049-1d4ae38a7d1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:54.718686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3dd221d1-e5cf-4b40-8049-1d4ae38a7d1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:54.828110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd221d1-e5cf-4b40-8049-1d4ae38a7d1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:54.830048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd221d1-e5cf-4b40-8049-1d4ae38a7d1d service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:54.852967 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 701/1403] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:46:57.065959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dc32c-294d-452d-ad97-c55e56602797 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:57.066533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dc32c-294d-452d-ad97-c55e56602797 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:57.066898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dc32c-294d-452d-ad97-c55e56602797 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:57.068295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 703/1404] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:57.073894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aed22-5522-42f7-a043-1bc6974f2fbc None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:57.074364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aed22-5522-42f7-a043-1bc6974f2fbc None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:57.074456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aed22-5522-42f7-a043-1bc6974f2fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:57.148391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-989aed22-5522-42f7-a043-1bc6974f2fbc None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:46:57.150757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-989aed22-5522-42f7-a043-1bc6974f2fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:57.193378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:57.289614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 702/1405] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:46:57.341029 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0dc32c-294d-452d-ad97-c55e56602797 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:46:57.342603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fc0dc32c-294d-452d-ad97-c55e56602797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:57.526311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0dc32c-294d-452d-ad97-c55e56602797 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:46:57.592155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:46:57.675824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5d2b6-a9f8-4d1b-85c5-df43bad1f6c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:46:57.677115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5d2b6-a9f8-4d1b-85c5-df43bad1f6c8 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:57.677599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5d2b6-a9f8-4d1b-85c5-df43bad1f6c8 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:57.677882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5d2b6-a9f8-4d1b-85c5-df43bad1f6c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:57.679577 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f9f5d2b6-a9f8-4d1b-85c5-df43bad1f6c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:46:57.791023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5d2b6-a9f8-4d1b-85c5-df43bad1f6c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:46:57.793272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5d2b6-a9f8-4d1b-85c5-df43bad1f6c8 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:46:57.816265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 704/1406] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:46:59.958264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d98790-7e2b-45ee-b053-f324afd13270 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:59.958834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d98790-7e2b-45ee-b053-f324afd13270 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:59.958834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d98790-7e2b-45ee-b053-f324afd13270 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:46:59.960365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 703/1407] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:46:59.965908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49329beb-378a-41c8-8239-80269aa9663e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:46:59.967254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49329beb-378a-41c8-8239-80269aa9663e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:46:59.967705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49329beb-378a-41c8-8239-80269aa9663e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:00.037735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:00.050446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-49329beb-378a-41c8-8239-80269aa9663e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:00.051708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-49329beb-378a-41c8-8239-80269aa9663e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:00.162812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 705/1408] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:00.212459 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d98790-7e2b-45ee-b053-f324afd13270 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:00.212459 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f0d98790-7e2b-45ee-b053-f324afd13270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:00.376101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d98790-7e2b-45ee-b053-f324afd13270 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:00.398418 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:00.513708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccf2c1-49a7-4996-be2f-d690833995e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:00.514644 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccf2c1-49a7-4996-be2f-d690833995e5 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:00.514926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccf2c1-49a7-4996-be2f-d690833995e5 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:00.515200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccf2c1-49a7-4996-be2f-d690833995e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:00.516935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f5ccf2c1-49a7-4996-be2f-d690833995e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:00.614910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccf2c1-49a7-4996-be2f-d690833995e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:00.616615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccf2c1-49a7-4996-be2f-d690833995e5 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:00.637696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 704/1409] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:10.160631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=44 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:10.634721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=90 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:17.063236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bb1c6-7f58-4e40-b894-5c7604778d0d None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:17.063819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bb1c6-7f58-4e40-b894-5c7604778d0d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:17.063819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bb1c6-7f58-4e40-b894-5c7604778d0d None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:17.065365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 706/1410] 10.222.0.85 () {54 vars in 918 bytes} [Tue Jul 19 20:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:17.071026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85141e12-8619-4edf-b4bf-c4ca1549a84b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:17.071556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85141e12-8619-4edf-b4bf-c4ca1549a84b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:17.071556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85141e12-8619-4edf-b4bf-c4ca1549a84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:17.146604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-85141e12-8619-4edf-b4bf-c4ca1549a84b None None] MFA Rules not processed for user `fffebfc51dba47f68263c0bcd6b18d22`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:17.147846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-85141e12-8619-4edf-b4bf-c4ca1549a84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:17.260689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 705/1411] 10.222.0.85 () {58 vars in 1015 bytes} [Tue Jul 19 20:47:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:17.275128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa7002-eea5-4c5b-8613-b457bbcac296 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:17.275449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa7002-eea5-4c5b-8613-b457bbcac296 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:17.275727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa7002-eea5-4c5b-8613-b457bbcac296 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:17.277690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 707/1412] 10.222.0.85 () {54 vars in 918 bytes} [Tue Jul 19 20:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:17.282411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537c87-f75a-4ed8-9047-7170fa18b759 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:17.282703 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537c87-f75a-4ed8-9047-7170fa18b759 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:17.282986 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537c87-f75a-4ed8-9047-7170fa18b759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:17.359180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:17.362339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e9537c87-f75a-4ed8-9047-7170fa18b759 None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:17.364197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e9537c87-f75a-4ed8-9047-7170fa18b759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:17.483224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 706/1413] 10.222.0.85 () {58 vars in 1015 bytes} [Tue Jul 19 20:47:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:20.133256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a3558-3c1d-4377-9bac-0ab7eae726d9 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:20.133849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a3558-3c1d-4377-9bac-0ab7eae726d9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:20.133849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a3558-3c1d-4377-9bac-0ab7eae726d9 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:20.135252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:20.140444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427a576-bc1e-4f2e-a1eb-af088410761f None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:20.140889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427a576-bc1e-4f2e-a1eb-af088410761f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:20.141037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427a576-bc1e-4f2e-a1eb-af088410761f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:20.207954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3427a576-bc1e-4f2e-a1eb-af088410761f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:20.209224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3427a576-bc1e-4f2e-a1eb-af088410761f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:20.265131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:20.318671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:20.363432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a3558-3c1d-4377-9bac-0ab7eae726d9 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:20.364982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b50a3558-3c1d-4377-9bac-0ab7eae726d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:20.512036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a3558-3c1d-4377-9bac-0ab7eae726d9 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:20.585072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:20.626387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d78931-a6af-4ee6-8a67-a0c55d130708 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:20.627043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d78931-a6af-4ee6-8a67-a0c55d130708 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:20.627329 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d78931-a6af-4ee6-8a67-a0c55d130708 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:20.627608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d78931-a6af-4ee6-8a67-a0c55d130708 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:20.629306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-36d78931-a6af-4ee6-8a67-a0c55d130708 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:20.719077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d78931-a6af-4ee6-8a67-a0c55d130708 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:20.720729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d78931-a6af-4ee6-8a67-a0c55d130708 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:20.739645 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 709/1416] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:21.665974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1f79c-0165-439a-b32f-12298e4932c3 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:21.665974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1f79c-0165-439a-b32f-12298e4932c3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:21.666629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1f79c-0165-439a-b32f-12298e4932c3 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:21.668524 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:47:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:21.677078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ee7d1-28ab-494e-b0b2-8fd134459896 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:21.677794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ee7d1-28ab-494e-b0b2-8fd134459896 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:21.677794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ee7d1-28ab-494e-b0b2-8fd134459896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:21.762595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-550ee7d1-28ab-494e-b0b2-8fd134459896 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:21.766273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-550ee7d1-28ab-494e-b0b2-8fd134459896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:21.794791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:21.874986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:22.028200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1f79c-0165-439a-b32f-12298e4932c3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:22.029723 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e9e1f79c-0165-439a-b32f-12298e4932c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:22.199721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1f79c-0165-439a-b32f-12298e4932c3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:22.242522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:22.321331 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d35e01-82e4-4bdc-bc80-aafa37208284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:22.321898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d35e01-82e4-4bdc-bc80-aafa37208284 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:22.322181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d35e01-82e4-4bdc-bc80-aafa37208284 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:22.322458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d35e01-82e4-4bdc-bc80-aafa37208284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:22.324172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-08d35e01-82e4-4bdc-bc80-aafa37208284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:22.423079 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d35e01-82e4-4bdc-bc80-aafa37208284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:22.425041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d35e01-82e4-4bdc-bc80-aafa37208284 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:22.445580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 709/1419] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:23.388287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c14a-00b9-4fd0-8e52-5f913197f2ad None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:23.388915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c14a-00b9-4fd0-8e52-5f913197f2ad None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:23.389054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362c14a-00b9-4fd0-8e52-5f913197f2ad None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:23.390580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:23.396363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8547de-8009-4f33-b9e4-f9b9e6cbfd5d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:23.397178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8547de-8009-4f33-b9e4-f9b9e6cbfd5d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:23.397352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8547de-8009-4f33-b9e4-f9b9e6cbfd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:23.458614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:23.486991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-0b8547de-8009-4f33-b9e4-f9b9e6cbfd5d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:23.488282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0b8547de-8009-4f33-b9e4-f9b9e6cbfd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:23.607383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:23.765112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362c14a-00b9-4fd0-8e52-5f913197f2ad None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:23.767374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a362c14a-00b9-4fd0-8e52-5f913197f2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:23.941571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362c14a-00b9-4fd0-8e52-5f913197f2ad None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:23.958526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:24.069776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516be60a-a5e5-49f2-b84c-363a295f334e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:24.070432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516be60a-a5e5-49f2-b84c-363a295f334e service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:24.070713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516be60a-a5e5-49f2-b84c-363a295f334e service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:24.070986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516be60a-a5e5-49f2-b84c-363a295f334e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:24.072740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-516be60a-a5e5-49f2-b84c-363a295f334e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:24.233936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516be60a-a5e5-49f2-b84c-363a295f334e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:24.237040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516be60a-a5e5-49f2-b84c-363a295f334e service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:24.265172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 712/1422] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:25.244479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a7760-8d4e-41e4-a5f1-50ec3a8e16e9 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:25.245130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a7760-8d4e-41e4-a5f1-50ec3a8e16e9 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:25.245130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a7760-8d4e-41e4-a5f1-50ec3a8e16e9 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:25.247003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:47:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:25.252544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25118750-94c7-42fd-b34b-e7e72b18f69f None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:25.253333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25118750-94c7-42fd-b34b-e7e72b18f69f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:25.253333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25118750-94c7-42fd-b34b-e7e72b18f69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:25.272858 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:25.337411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-25118750-94c7-42fd-b34b-e7e72b18f69f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:25.338685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-25118750-94c7-42fd-b34b-e7e72b18f69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:25.482422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:25.635217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a7760-8d4e-41e4-a5f1-50ec3a8e16e9 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:25.637619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cd2a7760-8d4e-41e4-a5f1-50ec3a8e16e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:25.756573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:25.795420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a7760-8d4e-41e4-a5f1-50ec3a8e16e9 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:25.920307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6db8c5-6dbd-4287-a33c-5b2ab4188045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:25.920992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6db8c5-6dbd-4287-a33c-5b2ab4188045 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:25.921278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6db8c5-6dbd-4287-a33c-5b2ab4188045 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:25.921554 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6db8c5-6dbd-4287-a33c-5b2ab4188045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:25.923269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bf6db8c5-6dbd-4287-a33c-5b2ab4188045 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:26.004629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:26.024690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6db8c5-6dbd-4287-a33c-5b2ab4188045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:26.026257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6db8c5-6dbd-4287-a33c-5b2ab4188045 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:26.046110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 712/1425] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:26.953808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef5a5d-c884-4e5f-a25c-7be3ba1786d3 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:26.954424 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef5a5d-c884-4e5f-a25c-7be3ba1786d3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:26.954424 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef5a5d-c884-4e5f-a25c-7be3ba1786d3 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:26.956033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:26.962084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c1b5-dc02-437e-b58f-c30b85fa7252 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:26.962692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c1b5-dc02-437e-b58f-c30b85fa7252 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:26.962692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c1b5-dc02-437e-b58f-c30b85fa7252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:27.054339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3ba9c1b5-dc02-437e-b58f-c30b85fa7252 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:27.055789 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3ba9c1b5-dc02-437e-b58f-c30b85fa7252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:27.178695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:47:27.342922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef5a5d-c884-4e5f-a25c-7be3ba1786d3 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:27.344943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-aaef5a5d-c884-4e5f-a25c-7be3ba1786d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:27.451901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:27.524687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef5a5d-c884-4e5f-a25c-7be3ba1786d3 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:27.749321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dff902-65be-4af0-87aa-d9d1c097517a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:27.749805 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dff902-65be-4af0-87aa-d9d1c097517a service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:27.750054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dff902-65be-4af0-87aa-d9d1c097517a service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:27.750762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dff902-65be-4af0-87aa-d9d1c097517a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:27.754237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-55dff902-65be-4af0-87aa-d9d1c097517a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:27.825343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:27.884306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dff902-65be-4af0-87aa-d9d1c097517a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:27.886727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dff902-65be-4af0-87aa-d9d1c097517a service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:27.914001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 715/1428] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:47:28.758378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353e697-63b1-400e-917e-babd41dd201a None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:28.759043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353e697-63b1-400e-917e-babd41dd201a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:28.759043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353e697-63b1-400e-917e-babd41dd201a None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:28.761197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:47:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:28.767198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3d178-4136-44f3-b5e3-5b377812f8f8 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:28.767965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3d178-4136-44f3-b5e3-5b377812f8f8 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:28.767965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3d178-4136-44f3-b5e3-5b377812f8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:28.856911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-90a3d178-4136-44f3-b5e3-5b377812f8f8 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:28.858172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-90a3d178-4136-44f3-b5e3-5b377812f8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:28.997468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:29.153944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353e697-63b1-400e-917e-babd41dd201a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:29.155408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e353e697-63b1-400e-917e-babd41dd201a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:29.212663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:29.350621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353e697-63b1-400e-917e-babd41dd201a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:29.515225 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227bbba-ac88-44cf-b7e3-ba01c6296769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:29.516624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227bbba-ac88-44cf-b7e3-ba01c6296769 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:29.517133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227bbba-ac88-44cf-b7e3-ba01c6296769 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:29.517630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227bbba-ac88-44cf-b7e3-ba01c6296769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:29.521224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2227bbba-ac88-44cf-b7e3-ba01c6296769 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:29.535168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:29.631229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227bbba-ac88-44cf-b7e3-ba01c6296769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:29.632933 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227bbba-ac88-44cf-b7e3-ba01c6296769 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:29.653653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 715/1431] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:47:30.508984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fb7d5-abf4-4c29-9df2-2dd463cb1db5 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:30.509535 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fb7d5-abf4-4c29-9df2-2dd463cb1db5 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:30.509535 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fb7d5-abf4-4c29-9df2-2dd463cb1db5 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:30.510899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:30.516571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64c1b-6e4f-4318-8834-05925e47170a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:30.517407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64c1b-6e4f-4318-8834-05925e47170a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:30.517407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64c1b-6e4f-4318-8834-05925e47170a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:30.605141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d4a64c1b-6e4f-4318-8834-05925e47170a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:30.606385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d4a64c1b-6e4f-4318-8834-05925e47170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:30.683630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:30.719159 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:30.882583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fb7d5-abf4-4c29-9df2-2dd463cb1db5 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:30.884158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fa9fb7d5-abf4-4c29-9df2-2dd463cb1db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:30.905122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:31.062928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fb7d5-abf4-4c29-9df2-2dd463cb1db5 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:31.162856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:31.201506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9279c1-22a5-4d58-bb46-8d32b331a2c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:31.202163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9279c1-22a5-4d58-bb46-8d32b331a2c8 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:31.202444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9279c1-22a5-4d58-bb46-8d32b331a2c8 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:31.202718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9279c1-22a5-4d58-bb46-8d32b331a2c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:31.204523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9c9279c1-22a5-4d58-bb46-8d32b331a2c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:31.335250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9279c1-22a5-4d58-bb46-8d32b331a2c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:31.338334 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9279c1-22a5-4d58-bb46-8d32b331a2c8 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:31.374070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 718/1434] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:32.211788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9af4a-0265-44dc-95ea-9f9fcdfdff0f None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:32.212675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9af4a-0265-44dc-95ea-9f9fcdfdff0f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:32.212675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9af4a-0265-44dc-95ea-9f9fcdfdff0f None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:32.214566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:47:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:47:32.220207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9fd8-bfd7-49a0-9e27-9d1add3b57ba None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:32.220674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9fd8-bfd7-49a0-9e27-9d1add3b57ba None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:32.220807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a9fd8-bfd7-49a0-9e27-9d1add3b57ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:32.317949 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-059a9fd8-bfd7-49a0-9e27-9d1add3b57ba None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:32.319233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-059a9fd8-bfd7-49a0-9e27-9d1add3b57ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:32.369712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:32.436110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:47:32.590101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9af4a-0265-44dc-95ea-9f9fcdfdff0f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:32.591593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-28f9af4a-0265-44dc-95ea-9f9fcdfdff0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:32.790704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9af4a-0265-44dc-95ea-9f9fcdfdff0f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:32.891879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:32.967545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cacd61-b502-40af-8f97-9ee69cb8f5fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:32.968261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cacd61-b502-40af-8f97-9ee69cb8f5fa service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:32.968630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cacd61-b502-40af-8f97-9ee69cb8f5fa service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:32.968953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cacd61-b502-40af-8f97-9ee69cb8f5fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:32.970741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-87cacd61-b502-40af-8f97-9ee69cb8f5fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:33.097686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cacd61-b502-40af-8f97-9ee69cb8f5fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:33.099424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cacd61-b502-40af-8f97-9ee69cb8f5fa service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:33.126597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 718/1437] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:47:34.041019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c31338-b290-4565-9ae1-00e3c1b53aa4 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:34.041637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c31338-b290-4565-9ae1-00e3c1b53aa4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:34.041637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c31338-b290-4565-9ae1-00e3c1b53aa4 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:34.043747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:47:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:34.049948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5741a-216d-489e-a7ca-a832ee832b37 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:34.049948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5741a-216d-489e-a7ca-a832ee832b37 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:34.051556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5741a-216d-489e-a7ca-a832ee832b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:34.269058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-11a5741a-216d-489e-a7ca-a832ee832b37 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:34.270351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-11a5741a-216d-489e-a7ca-a832ee832b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:34.272792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:34.391857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:47:34.543286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c31338-b290-4565-9ae1-00e3c1b53aa4 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:34.544991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f4c31338-b290-4565-9ae1-00e3c1b53aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:34.716954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c31338-b290-4565-9ae1-00e3c1b53aa4 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:34.751886 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:34.861134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b091e-a0ad-4070-b46b-7d0f11340bde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:34.861789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b091e-a0ad-4070-b46b-7d0f11340bde service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:34.862071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b091e-a0ad-4070-b46b-7d0f11340bde service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:34.862345 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b091e-a0ad-4070-b46b-7d0f11340bde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:34.864051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3f0b091e-a0ad-4070-b46b-7d0f11340bde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:34.965955 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b091e-a0ad-4070-b46b-7d0f11340bde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:34.967774 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b091e-a0ad-4070-b46b-7d0f11340bde service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:34.988361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 721/1440] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:37.141163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2f785-8296-4265-aaa4-925e01cd2727 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:37.142044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2f785-8296-4265-aaa4-925e01cd2727 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:37.142044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2f785-8296-4265-aaa4-925e01cd2727 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:37.144053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:47:37.150942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e03988-8fd9-46ba-9f11-d123bd797958 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:37.151546 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e03988-8fd9-46ba-9f11-d123bd797958 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:37.151848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e03988-8fd9-46ba-9f11-d123bd797958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:37.213265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:37.248609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-f3e03988-8fd9-46ba-9f11-d123bd797958 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:37.250118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f3e03988-8fd9-46ba-9f11-d123bd797958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:37.400418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:37.442517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2f785-8296-4265-aaa4-925e01cd2727 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:37.445328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e6c2f785-8296-4265-aaa4-925e01cd2727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:37.603426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2f785-8296-4265-aaa4-925e01cd2727 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:37.650510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:37.719106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9055d8-82d8-4a0d-bec5-aa28a8c565c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:37.720285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9055d8-82d8-4a0d-bec5-aa28a8c565c8 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:37.720680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9055d8-82d8-4a0d-bec5-aa28a8c565c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:37.721003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9055d8-82d8-4a0d-bec5-aa28a8c565c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:37.723534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0e9055d8-82d8-4a0d-bec5-aa28a8c565c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:37.890698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9055d8-82d8-4a0d-bec5-aa28a8c565c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:37.892169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9055d8-82d8-4a0d-bec5-aa28a8c565c8 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:37.914604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 721/1443] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:40.513060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b2db5-2800-47f8-a787-5e618da78744 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:40.513599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b2db5-2800-47f8-a787-5e618da78744 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:40.513599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b2db5-2800-47f8-a787-5e618da78744 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:40.514922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:47:40.520563 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e12a55-da9b-4414-983a-5437f0879283 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:40.521373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e12a55-da9b-4414-983a-5437f0879283 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:40.521373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e12a55-da9b-4414-983a-5437f0879283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:40.606205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-40e12a55-da9b-4414-983a-5437f0879283 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:40.607445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-40e12a55-da9b-4414-983a-5437f0879283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:40.620744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:40.730992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:40.775301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b2db5-2800-47f8-a787-5e618da78744 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:40.777711 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1c7b2db5-2800-47f8-a787-5e618da78744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:40.909444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b2db5-2800-47f8-a787-5e618da78744 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:40.924629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:41.006090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87dbda-bd79-456c-a392-ffd3e99b52fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:41.006737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87dbda-bd79-456c-a392-ffd3e99b52fd service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:41.007029 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87dbda-bd79-456c-a392-ffd3e99b52fd service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:41.007300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87dbda-bd79-456c-a392-ffd3e99b52fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:41.008998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9d87dbda-bd79-456c-a392-ffd3e99b52fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:41.132037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87dbda-bd79-456c-a392-ffd3e99b52fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:41.133437 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87dbda-bd79-456c-a392-ffd3e99b52fd service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:41.154947 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 724/1446] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:43.811482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1005e-2249-4392-b567-84c4c41c1a5a None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:43.812123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1005e-2249-4392-b567-84c4c41c1a5a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:43.812123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1005e-2249-4392-b567-84c4c41c1a5a None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:43.813523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:43.819851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cf7f-881c-4892-985e-da8dde5795a3 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:43.820453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cf7f-881c-4892-985e-da8dde5795a3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:43.820453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cf7f-881c-4892-985e-da8dde5795a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:43.880692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:43.902162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-1e00cf7f-881c-4892-985e-da8dde5795a3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:43.903426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1e00cf7f-881c-4892-985e-da8dde5795a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:44.021989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:44.067390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1005e-2249-4392-b567-84c4c41c1a5a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:44.068957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-36e1005e-2249-4392-b567-84c4c41c1a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:44.210308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1005e-2249-4392-b567-84c4c41c1a5a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:44.255606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:44.293900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d306e-f654-45ba-a70f-f2bec67d4e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:44.294619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d306e-f654-45ba-a70f-f2bec67d4e99 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:44.294943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d306e-f654-45ba-a70f-f2bec67d4e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:44.295542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d306e-f654-45ba-a70f-f2bec67d4e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:44.297410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fe7d306e-f654-45ba-a70f-f2bec67d4e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:44.441688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d306e-f654-45ba-a70f-f2bec67d4e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:44.443144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d306e-f654-45ba-a70f-f2bec67d4e99 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:44.464021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 724/1449] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:47.044208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c220364-9d03-404e-972b-b9b1349d20ed None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:47.045115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c220364-9d03-404e-972b-b9b1349d20ed None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:47.045115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c220364-9d03-404e-972b-b9b1349d20ed None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:47.046174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:47.053854 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6466bd-cd30-412a-a79f-e38219e38d6b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:47.054700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6466bd-cd30-412a-a79f-e38219e38d6b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:47.054928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6466bd-cd30-412a-a79f-e38219e38d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:47.135040 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-7f6466bd-cd30-412a-a79f-e38219e38d6b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:47.136904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7f6466bd-cd30-412a-a79f-e38219e38d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:47.165376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:47.268498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:47:47.313083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c220364-9d03-404e-972b-b9b1349d20ed None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:47.314808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6c220364-9d03-404e-972b-b9b1349d20ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:47.435739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c220364-9d03-404e-972b-b9b1349d20ed None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:47.461842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:47.542203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98dbab-8d86-4a51-a702-2c8fae8b7bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:47.543517 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98dbab-8d86-4a51-a702-2c8fae8b7bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:47.544022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98dbab-8d86-4a51-a702-2c8fae8b7bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:47.544644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98dbab-8d86-4a51-a702-2c8fae8b7bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:47.547958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4d98dbab-8d86-4a51-a702-2c8fae8b7bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:47.726654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98dbab-8d86-4a51-a702-2c8fae8b7bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:47.728291 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98dbab-8d86-4a51-a702-2c8fae8b7bf1 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:47.752298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 727/1452] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:50.346651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb381a6-5258-4489-9395-e1bb04124aa9 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:50.347212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb381a6-5258-4489-9395-e1bb04124aa9 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:50.347212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb381a6-5258-4489-9395-e1bb04124aa9 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:50.348774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:47:50.355128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3d7c3-0bd7-44d9-98b6-13be9435011f None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:50.355682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3d7c3-0bd7-44d9-98b6-13be9435011f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:50.356079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3d7c3-0bd7-44d9-98b6-13be9435011f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:50.449554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:50.453510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c4f3d7c3-0bd7-44d9-98b6-13be9435011f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:50.455572 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c4f3d7c3-0bd7-44d9-98b6-13be9435011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:50.567503 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:50.610992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb381a6-5258-4489-9395-e1bb04124aa9 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:50.612734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3cb381a6-5258-4489-9395-e1bb04124aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:50.739848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb381a6-5258-4489-9395-e1bb04124aa9 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:50.796243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:50.824310 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b16a7b-11d1-4406-8cc6-1390943497da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:50.824905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b16a7b-11d1-4406-8cc6-1390943497da service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:50.825188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b16a7b-11d1-4406-8cc6-1390943497da service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:50.825461 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b16a7b-11d1-4406-8cc6-1390943497da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:50.827171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-19b16a7b-11d1-4406-8cc6-1390943497da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:50.955043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b16a7b-11d1-4406-8cc6-1390943497da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:50.957027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b16a7b-11d1-4406-8cc6-1390943497da service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:50.980072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 727/1455] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:53.246343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ebb1-7f8a-4162-8d11-74eace166694 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:53.246979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ebb1-7f8a-4162-8d11-74eace166694 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:53.246979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550ebb1-7f8a-4162-8d11-74eace166694 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:53.248635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:53.254774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11821c41-f3a4-4a0c-ae7c-a105cc0929fd None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:53.255372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11821c41-f3a4-4a0c-ae7c-a105cc0929fd None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:53.255522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11821c41-f3a4-4a0c-ae7c-a105cc0929fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:53.335477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-11821c41-f3a4-4a0c-ae7c-a105cc0929fd None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:53.336835 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-11821c41-f3a4-4a0c-ae7c-a105cc0929fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:53.373411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:53.459892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:53.501203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550ebb1-7f8a-4162-8d11-74eace166694 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:53.502942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0550ebb1-7f8a-4162-8d11-74eace166694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:53.629510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550ebb1-7f8a-4162-8d11-74eace166694 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:53.661297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:53.732620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c4bd6-8b0e-4dfc-9810-78e2bfd563a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:53.733251 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c4bd6-8b0e-4dfc-9810-78e2bfd563a2 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:53.733577 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c4bd6-8b0e-4dfc-9810-78e2bfd563a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:53.734014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c4bd6-8b0e-4dfc-9810-78e2bfd563a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:53.736166 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0a8c4bd6-8b0e-4dfc-9810-78e2bfd563a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:53.865470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c4bd6-8b0e-4dfc-9810-78e2bfd563a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:53.866929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c4bd6-8b0e-4dfc-9810-78e2bfd563a2 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:53.888171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 730/1458] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:47:56.265556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f04454-cd7e-417e-8ce3-7273378422e2 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:56.266432 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f04454-cd7e-417e-8ce3-7273378422e2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:56.266432 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f04454-cd7e-417e-8ce3-7273378422e2 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:56.268052 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:47:56.274895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bc51d-8d06-4264-857b-557f416eb3c3 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:56.275451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bc51d-8d06-4264-857b-557f416eb3c3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:56.275451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bc51d-8d06-4264-857b-557f416eb3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:56.355005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-370bc51d-8d06-4264-857b-557f416eb3c3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:56.356292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-370bc51d-8d06-4264-857b-557f416eb3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:56.368508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:56.474613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:56.515303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f04454-cd7e-417e-8ce3-7273378422e2 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:56.517164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-23f04454-cd7e-417e-8ce3-7273378422e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:56.630703 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f04454-cd7e-417e-8ce3-7273378422e2 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:56.694106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:56.714501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b97db-70d3-4f86-91c2-7854fb60c0b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:56.715201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b97db-70d3-4f86-91c2-7854fb60c0b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:56.715480 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b97db-70d3-4f86-91c2-7854fb60c0b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:56.715762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b97db-70d3-4f86-91c2-7854fb60c0b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:56.717499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f94b97db-70d3-4f86-91c2-7854fb60c0b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:56.839287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b97db-70d3-4f86-91c2-7854fb60c0b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:56.840767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b97db-70d3-4f86-91c2-7854fb60c0b2 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:56.862507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 730/1461] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:47:59.121815 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f989f9c-bc47-47d3-bbff-c0f22c956c41 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:59.122400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f989f9c-bc47-47d3-bbff-c0f22c956c41 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:59.122507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f989f9c-bc47-47d3-bbff-c0f22c956c41 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:59.123976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:47:59.131088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eb9b8-760a-4827-b4ee-80bfa6dbdad3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:59.131636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eb9b8-760a-4827-b4ee-80bfa6dbdad3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:59.131957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eb9b8-760a-4827-b4ee-80bfa6dbdad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:59.218126 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e56eb9b8-760a-4827-b4ee-80bfa6dbdad3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:47:59.220712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e56eb9b8-760a-4827-b4ee-80bfa6dbdad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:59.270890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:59.345173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:47:59.412531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f989f9c-bc47-47d3-bbff-c0f22c956c41 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:47:59.414403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9f989f9c-bc47-47d3-bbff-c0f22c956c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:59.532003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f989f9c-bc47-47d3-bbff-c0f22c956c41 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:47:59.572131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:47:59.609793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cb03c-05bd-495d-bfcd-da33ee854a73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:47:59.610478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cb03c-05bd-495d-bfcd-da33ee854a73 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:47:59.610764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cb03c-05bd-495d-bfcd-da33ee854a73 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:47:59.611044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cb03c-05bd-495d-bfcd-da33ee854a73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:47:59.612800 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e57cb03c-05bd-495d-bfcd-da33ee854a73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:47:59.730323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57cb03c-05bd-495d-bfcd-da33ee854a73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:47:59.731974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57cb03c-05bd-495d-bfcd-da33ee854a73 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:47:59.752683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 733/1464] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:47:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:48:01.918184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf65ea-c42f-4c33-9ad9-62341faeb579 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:01.919007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf65ea-c42f-4c33-9ad9-62341faeb579 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:01.919200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf65ea-c42f-4c33-9ad9-62341faeb579 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:01.920698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:48:01.927363 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10dec1-a4d5-49f4-a80e-c78d7fc6c48b None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:01.927895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10dec1-a4d5-49f4-a80e-c78d7fc6c48b None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:01.928027 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10dec1-a4d5-49f4-a80e-c78d7fc6c48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:02.000095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-af10dec1-a4d5-49f4-a80e-c78d7fc6c48b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:02.001474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-af10dec1-a4d5-49f4-a80e-c78d7fc6c48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:02.020819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:02.112491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:48:02.151485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf65ea-c42f-4c33-9ad9-62341faeb579 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:02.153144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-43cf65ea-c42f-4c33-9ad9-62341faeb579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:02.281542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf65ea-c42f-4c33-9ad9-62341faeb579 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:02.362442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:02.376244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6608ea3-7ba1-4e2e-8e7d-c9d9a3ba9ccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:02.377457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6608ea3-7ba1-4e2e-8e7d-c9d9a3ba9ccb service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:02.377912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6608ea3-7ba1-4e2e-8e7d-c9d9a3ba9ccb service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:02.378343 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6608ea3-7ba1-4e2e-8e7d-c9d9a3ba9ccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:02.381378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e6608ea3-7ba1-4e2e-8e7d-c9d9a3ba9ccb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:02.534427 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6608ea3-7ba1-4e2e-8e7d-c9d9a3ba9ccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:02.535975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6608ea3-7ba1-4e2e-8e7d-c9d9a3ba9ccb service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:02.556557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 733/1467] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:48:04.683169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a53cb-6c18-4fc8-a5f9-d9b25cee1e72 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:04.684028 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a53cb-6c18-4fc8-a5f9-d9b25cee1e72 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:04.684250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a53cb-6c18-4fc8-a5f9-d9b25cee1e72 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:04.686477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:48:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:48:04.694301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a253c9-7ba0-4463-b208-f4bf5279f4aa None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:04.695199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a253c9-7ba0-4463-b208-f4bf5279f4aa None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:04.695199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a253c9-7ba0-4463-b208-f4bf5279f4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:04.772520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-07a253c9-7ba0-4463-b208-f4bf5279f4aa None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:04.773768 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-07a253c9-7ba0-4463-b208-f4bf5279f4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:04.830986 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:04.886408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:04.926306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a53cb-6c18-4fc8-a5f9-d9b25cee1e72 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:04.927913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1c2a53cb-6c18-4fc8-a5f9-d9b25cee1e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:05.052898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a53cb-6c18-4fc8-a5f9-d9b25cee1e72 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:05.107781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:05.139530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f215d-d804-409b-8e4c-f4263c745dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:05.140622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f215d-d804-409b-8e4c-f4263c745dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:05.141092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f215d-d804-409b-8e4c-f4263c745dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:05.141592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f215d-d804-409b-8e4c-f4263c745dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:05.145011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-089f215d-d804-409b-8e4c-f4263c745dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:05.298255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f215d-d804-409b-8e4c-f4263c745dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:05.299771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f215d-d804-409b-8e4c-f4263c745dc8 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:05.323798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 736/1470] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:14.883707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=22 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:15.320423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=66 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:27.061480 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee17eb-c87d-48f3-b24a-cc64ab810d84 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:27.062390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee17eb-c87d-48f3-b24a-cc64ab810d84 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:27.062390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee17eb-c87d-48f3-b24a-cc64ab810d84 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:27.063414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 735/1471] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:48:27.069694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a07c45-e824-46a2-b90e-b480b0c74b9b None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:27.070253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a07c45-e824-46a2-b90e-b480b0c74b9b None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:27.070253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a07c45-e824-46a2-b90e-b480b0c74b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:27.143585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-42a07c45-e824-46a2-b90e-b480b0c74b9b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:27.144867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-42a07c45-e824-46a2-b90e-b480b0c74b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:27.259403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 737/1472] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:48:27.357713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1b096-3389-4a44-925c-de78c515e8e5 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:27.358089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1b096-3389-4a44-925c-de78c515e8e5 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:27.358491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1b096-3389-4a44-925c-de78c515e8e5 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:27.360536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 736/1473] 10.222.0.85 () {54 vars in 966 bytes} [Tue Jul 19 20:48:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:48:27.368768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecaca9-a7b4-49f0-b7a9-f694277f98b2 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:27.369232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecaca9-a7b4-49f0-b7a9-f694277f98b2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:27.369653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecaca9-a7b4-49f0-b7a9-f694277f98b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:27.476622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-a4ecaca9-a7b4-49f0-b7a9-f694277f98b2 None None] MFA Rules not processed for user `e35e7720e3bf4c9e80bdefa6a1cb8296`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:27.477873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a4ecaca9-a7b4-49f0-b7a9-f694277f98b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:27.501330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:27.566636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 738/1474] 10.222.0.85 () {58 vars in 1063 bytes} [Tue Jul 19 20:48:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:27.575058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1b096-3389-4a44-925c-de78c515e8e5 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:27.577122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e7c1b096-3389-4a44-925c-de78c515e8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:27.710779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1b096-3389-4a44-925c-de78c515e8e5 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:27.860285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325a486-a79b-4ae0-a4ca-2f7d30069759 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35e7720e3bf4c9e80bdefa6a1cb8296', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:27.861467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325a486-a79b-4ae0-a4ca-2f7d30069759 service glance] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:27.861972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325a486-a79b-4ae0-a4ca-2f7d30069759 service glance] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:27.862468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325a486-a79b-4ae0-a4ca-2f7d30069759 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:27.865931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3325a486-a79b-4ae0-a4ca-2f7d30069759 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:27.946515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:28.029890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325a486-a79b-4ae0-a4ca-2f7d30069759 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:28.031237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325a486-a79b-4ae0-a4ca-2f7d30069759 service glance] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:28.055548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 737/1475] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:30.269089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82e995-b18d-4f61-885d-e61bab248da3 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:30.269664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82e995-b18d-4f61-885d-e61bab248da3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:30.269664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82e995-b18d-4f61-885d-e61bab248da3 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:30.271245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 739/1476] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:48:30.277433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb160af7-745d-42d7-8ef6-7157f09f3545 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:30.277967 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb160af7-745d-42d7-8ef6-7157f09f3545 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:30.277967 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb160af7-745d-42d7-8ef6-7157f09f3545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:30.352739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-eb160af7-745d-42d7-8ef6-7157f09f3545 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:30.354050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-eb160af7-745d-42d7-8ef6-7157f09f3545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:30.467135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 738/1477] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:48:30.493096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82e995-b18d-4f61-885d-e61bab248da3 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:30.495282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4f82e995-b18d-4f61-885d-e61bab248da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:30.638848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82e995-b18d-4f61-885d-e61bab248da3 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:30.699348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:30.750521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cf9b4-9ae7-4325-ad6c-b3e5793914fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:30.751072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cf9b4-9ae7-4325-ad6c-b3e5793914fc admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:30.751353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cf9b4-9ae7-4325-ad6c-b3e5793914fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:30.751630 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cf9b4-9ae7-4325-ad6c-b3e5793914fc admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:30.752496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cf9b4-9ae7-4325-ad6c-b3e5793914fc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:30.754218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: WARNING py.warnings [None req-d16cf9b4-9ae7-4325-ad6c-b3e5793914fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:48:30.754218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: warnings.warn(msg) Jul 19 20:48:30.754218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: Jul 19 20:48:30.755366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cf9b4-9ae7-4325-ad6c-b3e5793914fc admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:30.776903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 740/1478] 10.222.0.85 () {56 vars in 1108 bytes} [Tue Jul 19 20:48:30 2022] GET /identity/v3/projects => generated 2553 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:48:32.890919 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0ee90-91e3-4ef7-8b9c-6fb6100910f3 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:32.891699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0ee90-91e3-4ef7-8b9c-6fb6100910f3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:32.891699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0ee90-91e3-4ef7-8b9c-6fb6100910f3 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:32.892889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 739/1479] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:48:32.898476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427132a-e0f3-4ed2-81df-c065db9e6e13 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:32.898995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427132a-e0f3-4ed2-81df-c065db9e6e13 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:32.899104 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427132a-e0f3-4ed2-81df-c065db9e6e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:32.972784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0427132a-e0f3-4ed2-81df-c065db9e6e13 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:32.974023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0427132a-e0f3-4ed2-81df-c065db9e6e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:33.076237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 741/1480] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:33.121964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0ee90-91e3-4ef7-8b9c-6fb6100910f3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:33.123973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-eec0ee90-91e3-4ef7-8b9c-6fb6100910f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:33.134240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:33.303743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0ee90-91e3-4ef7-8b9c-6fb6100910f3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:33.404997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:33.447463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd824eca-3423-4947-846c-facfa4dbec10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:33.448124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd824eca-3423-4947-846c-facfa4dbec10 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:33.448444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd824eca-3423-4947-846c-facfa4dbec10 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:33.448723 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd824eca-3423-4947-846c-facfa4dbec10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:33.450402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dd824eca-3423-4947-846c-facfa4dbec10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:33.556779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd824eca-3423-4947-846c-facfa4dbec10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:33.558339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd824eca-3423-4947-846c-facfa4dbec10 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:33.580788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 740/1481] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:35.980279 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d04766-5404-46ad-ab35-53c0abc1a8dd None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:35.981196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d04766-5404-46ad-ab35-53c0abc1a8dd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:35.981196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d04766-5404-46ad-ab35-53c0abc1a8dd None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:35.982801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 742/1482] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:48:35.989338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012f155-f615-4de2-adb4-246a4158928f None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:35.989908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012f155-f615-4de2-adb4-246a4158928f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:35.989908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012f155-f615-4de2-adb4-246a4158928f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:36.064600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c012f155-f615-4de2-adb4-246a4158928f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:36.065871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c012f155-f615-4de2-adb4-246a4158928f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:36.120975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:36.192164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 741/1483] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:36.236387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d04766-5404-46ad-ab35-53c0abc1a8dd None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:36.237850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f7d04766-5404-46ad-ab35-53c0abc1a8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:36.266903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:36.395125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d04766-5404-46ad-ab35-53c0abc1a8dd None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:36.488510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedda16-492e-4581-a6b5-5ef5c904eea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:36.489158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedda16-492e-4581-a6b5-5ef5c904eea8 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:36.489487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedda16-492e-4581-a6b5-5ef5c904eea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:36.489804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedda16-492e-4581-a6b5-5ef5c904eea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:36.491615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3aedda16-492e-4581-a6b5-5ef5c904eea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:36.585219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:36.630336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedda16-492e-4581-a6b5-5ef5c904eea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:36.631937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedda16-492e-4581-a6b5-5ef5c904eea8 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:36.654275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 743/1484] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:48:39.139908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf489e3c-6c27-402a-b070-b3838f3b6ccb None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:39.140538 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf489e3c-6c27-402a-b070-b3838f3b6ccb None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:39.140538 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf489e3c-6c27-402a-b070-b3838f3b6ccb None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:39.142013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 742/1485] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:48:39.149267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434d0b1-6a5a-4143-8e2a-ef3a339e2835 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:39.149988 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434d0b1-6a5a-4143-8e2a-ef3a339e2835 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:39.150111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434d0b1-6a5a-4143-8e2a-ef3a339e2835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:39.250892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-1434d0b1-6a5a-4143-8e2a-ef3a339e2835 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:39.252150 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1434d0b1-6a5a-4143-8e2a-ef3a339e2835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:39.364755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 744/1486] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:39.416590 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf489e3c-6c27-402a-b070-b3838f3b6ccb None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:39.418275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bf489e3c-6c27-402a-b070-b3838f3b6ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:39.580718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf489e3c-6c27-402a-b070-b3838f3b6ccb None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:39.700903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:39.704385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54c2d4-b8b6-4e92-80b8-d5f31481d479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:39.705537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54c2d4-b8b6-4e92-80b8-d5f31481d479 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:39.706104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54c2d4-b8b6-4e92-80b8-d5f31481d479 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:39.706726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54c2d4-b8b6-4e92-80b8-d5f31481d479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:39.708926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fc54c2d4-b8b6-4e92-80b8-d5f31481d479 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:39.841786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54c2d4-b8b6-4e92-80b8-d5f31481d479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:39.843263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54c2d4-b8b6-4e92-80b8-d5f31481d479 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:39.865022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 743/1487] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:42.224822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344571ba-efcc-4525-82fe-9ec512e5cdd3 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:42.225428 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344571ba-efcc-4525-82fe-9ec512e5cdd3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:42.225533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344571ba-efcc-4525-82fe-9ec512e5cdd3 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:42.227001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 745/1488] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:48:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:48:42.234214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4fe2d-b5a5-46bf-be09-e543eb4d9f01 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:42.234991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4fe2d-b5a5-46bf-be09-e543eb4d9f01 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:42.234991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4fe2d-b5a5-46bf-be09-e543eb4d9f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:42.311071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-92c4fe2d-b5a5-46bf-be09-e543eb4d9f01 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:42.313226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-92c4fe2d-b5a5-46bf-be09-e543eb4d9f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:42.411352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:42.466888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 744/1489] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:42.513664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344571ba-efcc-4525-82fe-9ec512e5cdd3 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:42.517846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-344571ba-efcc-4525-82fe-9ec512e5cdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:42.578008 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:42.674795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344571ba-efcc-4525-82fe-9ec512e5cdd3 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:42.772727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3097ff0-7a19-4857-9775-e0922a45619f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:42.773307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3097ff0-7a19-4857-9775-e0922a45619f service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:42.773678 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3097ff0-7a19-4857-9775-e0922a45619f service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:42.773959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3097ff0-7a19-4857-9775-e0922a45619f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:42.776247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f3097ff0-7a19-4857-9775-e0922a45619f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:42.875952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:42.911638 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3097ff0-7a19-4857-9775-e0922a45619f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:42.913130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3097ff0-7a19-4857-9775-e0922a45619f service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:42.933746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 746/1490] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:48:45.386445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f96172-674c-4bf4-92ae-0d08629d61c7 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:45.387254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f96172-674c-4bf4-92ae-0d08629d61c7 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:45.387417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f96172-674c-4bf4-92ae-0d08629d61c7 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:45.390255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 745/1491] 10.222.0.85 () {54 vars in 926 bytes} [Tue Jul 19 20:48:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:48:45.399948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f3b8e-bd7f-4d82-b537-6451f77ee86e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:45.400565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f3b8e-bd7f-4d82-b537-6451f77ee86e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:45.401052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f3b8e-bd7f-4d82-b537-6451f77ee86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:45.517219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-410f3b8e-bd7f-4d82-b537-6451f77ee86e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:48:45.518480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-410f3b8e-bd7f-4d82-b537-6451f77ee86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:45.694486 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 747/1492] 10.222.0.85 () {58 vars in 1023 bytes} [Tue Jul 19 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:48:45.766947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f96172-674c-4bf4-92ae-0d08629d61c7 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:48:45.768735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-24f96172-674c-4bf4-92ae-0d08629d61c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:45.906134 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f96172-674c-4bf4-92ae-0d08629d61c7 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:48:46.051610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bb12a-5bd0-44d5-a079-1b808823b218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:48:46.052644 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bb12a-5bd0-44d5-a079-1b808823b218 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:48:46.053108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bb12a-5bd0-44d5-a079-1b808823b218 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:48:46.053558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bb12a-5bd0-44d5-a079-1b808823b218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:48:46.057209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cc2bb12a-5bd0-44d5-a079-1b808823b218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:48:46.075216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:46.226044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bb12a-5bd0-44d5-a079-1b808823b218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:48:46.227653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bb12a-5bd0-44d5-a079-1b808823b218 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:48:46.248399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 746/1493] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:48:55.689129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=82 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:48:56.245861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=50 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:17.271116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b1b7b-9e2a-489e-843b-5f97f1dc85a8 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:17.271712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b1b7b-9e2a-489e-843b-5f97f1dc85a8 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:17.271712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b1b7b-9e2a-489e-843b-5f97f1dc85a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:17.352865 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-b13b1b7b-9e2a-489e-843b-5f97f1dc85a8 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:17.354103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b13b1b7b-9e2a-489e-843b-5f97f1dc85a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:17.459351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 748/1494] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:17.468212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677290b8-0567-4f9e-a236-7d3624636919 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:17.468757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677290b8-0567-4f9e-a236-7d3624636919 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:17.468757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677290b8-0567-4f9e-a236-7d3624636919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:17.542045 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-677290b8-0567-4f9e-a236-7d3624636919 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:17.543255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-677290b8-0567-4f9e-a236-7d3624636919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:17.643585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 747/1495] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:17.648820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b1b7b-9e2a-489e-843b-5f97f1dc85a8 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:17.650197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b13b1b7b-9e2a-489e-843b-5f97f1dc85a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:17.661589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b1b7b-9e2a-489e-843b-5f97f1dc85a8 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:17.723295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:17.757041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204876b-887d-4160-8056-7d077618d653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:17.758260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204876b-887d-4160-8056-7d077618d653 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:17.758545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204876b-887d-4160-8056-7d077618d653 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:17.758820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204876b-887d-4160-8056-7d077618d653 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:17.759541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204876b-887d-4160-8056-7d077618d653 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:50:17.759848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204876b-887d-4160-8056-7d077618d653 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:17.760929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: WARNING py.warnings [None req-a204876b-887d-4160-8056-7d077618d653 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:50:17.760929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: warnings.warn(msg) Jul 19 20:50:17.760929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: Jul 19 20:50:17.761815 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204876b-887d-4160-8056-7d077618d653 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:17.777407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 749/1496] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:50:17.782447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677290b8-0567-4f9e-a236-7d3624636919 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:17.791875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677290b8-0567-4f9e-a236-7d3624636919 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:17.857239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:17.893015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912cb11-7fab-43ae-a191-7924b2125060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:17.893684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912cb11-7fab-43ae-a191-7924b2125060 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:17.893968 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912cb11-7fab-43ae-a191-7924b2125060 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:17.894247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912cb11-7fab-43ae-a191-7924b2125060 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:17.894937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912cb11-7fab-43ae-a191-7924b2125060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:17.896704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: WARNING py.warnings [None req-2912cb11-7fab-43ae-a191-7924b2125060 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:50:17.896704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: warnings.warn(msg) Jul 19 20:50:17.896704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: Jul 19 20:50:17.898036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912cb11-7fab-43ae-a191-7924b2125060 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:17.957825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 748/1497] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:50:17 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:50:17.962002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204876b-887d-4160-8056-7d077618d653 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:17.963571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a204876b-887d-4160-8056-7d077618d653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:17.997508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204876b-887d-4160-8056-7d077618d653 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.092023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e09c7-fea7-4bb4-9e79-4f2afe1bc4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.092717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e09c7-fea7-4bb4-9e79-4f2afe1bc4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.092997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e09c7-fea7-4bb4-9e79-4f2afe1bc4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.093268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e09c7-fea7-4bb4-9e79-4f2afe1bc4f1 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.093961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e09c7-fea7-4bb4-9e79-4f2afe1bc4f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.094958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: WARNING py.warnings [None req-5d5e09c7-fea7-4bb4-9e79-4f2afe1bc4f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:50:18.094958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: warnings.warn(msg) Jul 19 20:50:18.094958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: Jul 19 20:50:18.096151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e09c7-fea7-4bb4-9e79-4f2afe1bc4f1 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.137844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 750/1498] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:50:17 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:50:18.141979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912cb11-7fab-43ae-a191-7924b2125060 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.143526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2912cb11-7fab-43ae-a191-7924b2125060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:18.207125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912cb11-7fab-43ae-a191-7924b2125060 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.294934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:18.308391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b478b-dc6c-4421-8193-e483d0900660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.309058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b478b-dc6c-4421-8193-e483d0900660 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.309335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b478b-dc6c-4421-8193-e483d0900660 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.309608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b478b-dc6c-4421-8193-e483d0900660 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.310328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b478b-dc6c-4421-8193-e483d0900660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.311341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: WARNING py.warnings [None req-8a6b478b-dc6c-4421-8193-e483d0900660 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:50:18.311341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: warnings.warn(msg) Jul 19 20:50:18.311341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: Jul 19 20:50:18.312514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b478b-dc6c-4421-8193-e483d0900660 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.325047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 749/1499] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:18.329428 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e09c7-fea7-4bb4-9e79-4f2afe1bc4f1 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.330979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-5d5e09c7-fea7-4bb4-9e79-4f2afe1bc4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:18.342468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e09c7-fea7-4bb4-9e79-4f2afe1bc4f1 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.368145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:18.437731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af322000-0d24-44d1-90d2-1eb81d6f1230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.438395 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af322000-0d24-44d1-90d2-1eb81d6f1230 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.438673 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af322000-0d24-44d1-90d2-1eb81d6f1230 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.438944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af322000-0d24-44d1-90d2-1eb81d6f1230 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.439663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af322000-0d24-44d1-90d2-1eb81d6f1230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.440701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: WARNING py.warnings [None req-af322000-0d24-44d1-90d2-1eb81d6f1230 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:50:18.440701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: warnings.warn(msg) Jul 19 20:50:18.440701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: Jul 19 20:50:18.441593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af322000-0d24-44d1-90d2-1eb81d6f1230 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.454480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 751/1500] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:50:18.458828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b478b-dc6c-4421-8193-e483d0900660 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.461010 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8a6b478b-dc6c-4421-8193-e483d0900660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:18.494304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b478b-dc6c-4421-8193-e483d0900660 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:18.586907 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2af8f-ab10-41a8-bf4f-ce58af12d4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:18.587740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2af8f-ab10-41a8-bf4f-ce58af12d4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.588022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2af8f-ab10-41a8-bf4f-ce58af12d4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.588300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2af8f-ab10-41a8-bf4f-ce58af12d4b2 admin admin] PATH_INFO: `/v3/projects/d6f5d35ce2b74281984ba9b20ddcc968/users/702c79dd80304a9d84c5513c1243f592/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.610724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2af8f-ab10-41a8-bf4f-ce58af12d4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f5d35ce2b74281984ba9b20ddcc968, user_id=702c79dd80304a9d84c5513c1243f592, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:18.611927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: WARNING py.warnings [None req-9ec2af8f-ab10-41a8-bf4f-ce58af12d4b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:50:18.611927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: warnings.warn(msg) Jul 19 20:50:18.611927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: Jul 19 20:50:18.613894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2af8f-ab10-41a8-bf4f-ce58af12d4b2 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:18.639312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 750/1501] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:50:18 2022] PUT /identity/v3/projects/d6f5d35ce2b74281984ba9b20ddcc968/users/702c79dd80304a9d84c5513c1243f592/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:18.651576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dddc6-7589-41ac-9ce2-acb24b5c8a49 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.651863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dddc6-7589-41ac-9ce2-acb24b5c8a49 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.652148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dddc6-7589-41ac-9ce2-acb24b5c8a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.710482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-649dddc6-7589-41ac-9ce2-acb24b5c8a49 None None] MFA Rules not processed for user `702c79dd80304a9d84c5513c1243f592`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:18.711698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-649dddc6-7589-41ac-9ce2-acb24b5c8a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:18.788178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 752/1502] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:50:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:18.801761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ce660-0f21-45ad-b56d-174d08f347ca None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.802048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ce660-0f21-45ad-b56d-174d08f347ca None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.802325 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ce660-0f21-45ad-b56d-174d08f347ca None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.803682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 751/1503] 10.222.0.85 () {54 vars in 964 bytes} [Tue Jul 19 20:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:50:18.809294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21756adb-4145-4908-8c98-a213f1f68a5f None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:18.809707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21756adb-4145-4908-8c98-a213f1f68a5f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:18.810423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21756adb-4145-4908-8c98-a213f1f68a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:18.840781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:18.886140 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-21756adb-4145-4908-8c98-a213f1f68a5f None None] MFA Rules not processed for user `81411204bc9f4210880d807b41da9643`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:18.887382 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-21756adb-4145-4908-8c98-a213f1f68a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:18.965439 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 753/1504] 10.222.0.85 () {58 vars in 1061 bytes} [Tue Jul 19 20:50:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:18.973308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ce660-0f21-45ad-b56d-174d08f347ca None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:18.974846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-857ce660-0f21-45ad-b56d-174d08f347ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:19.025034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:19.080850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ce660-0f21-45ad-b56d-174d08f347ca None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:19.154140 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a1d4e-f769-45fb-bba3-b2c3289f74c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:19.154700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a1d4e-f769-45fb-bba3-b2c3289f74c4 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.154980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a1d4e-f769-45fb-bba3-b2c3289f74c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.155252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a1d4e-f769-45fb-bba3-b2c3289f74c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.157047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c71a1d4e-f769-45fb-bba3-b2c3289f74c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:19.265094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a1d4e-f769-45fb-bba3-b2c3289f74c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:19.266522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a1d4e-f769-45fb-bba3-b2c3289f74c4 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:19.270108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:19.286216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 752/1505] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:19.331473 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed796943-bb2b-4975-b49a-c71358486f95 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.332002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed796943-bb2b-4975-b49a-c71358486f95 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.332002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed796943-bb2b-4975-b49a-c71358486f95 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.333349 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 754/1506] 10.222.0.85 () {52 vars in 829 bytes} [Tue Jul 19 20:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:50:19.337906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a1d4e-f769-45fb-bba3-b2c3289f74c4 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:19.339134 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c71a1d4e-f769-45fb-bba3-b2c3289f74c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:19.445993 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a1d4e-f769-45fb-bba3-b2c3289f74c4 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:19.540604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:19.541484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.541761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.542032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] PATH_INFO: `/v3/users/702c79dd80304a9d84c5513c1243f592` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.554796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] RBAC: Authorizing `identity:delete_user(user_id=702c79dd80304a9d84c5513c1243f592)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:19.555857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: WARNING py.warnings [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:50:19.555857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: warnings.warn(msg) Jul 19 20:50:19.555857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: Jul 19 20:50:19.557115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:19.567463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:19.626894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702c79dd80304a9d84c5513c1243f592'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:19.636790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702c79dd80304a9d84c5513c1243f592'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:19.649919 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702c79dd80304a9d84c5513c1243f592'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:19.650384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702c79dd80304a9d84c5513c1243f592'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:19.668807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702c79dd80304a9d84c5513c1243f592'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:19.675227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 753/1507] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:50:19 2022] DELETE /identity/v3/users/702c79dd80304a9d84c5513c1243f592 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:19.679178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47c811-5ab6-4174-8c98-bc9c5770aa4e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:19.679473 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47c811-5ab6-4174-8c98-bc9c5770aa4e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:19.679752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47c811-5ab6-4174-8c98-bc9c5770aa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:19.737676 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-6c47c811-5ab6-4174-8c98-bc9c5770aa4e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:50:19.738797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6c47c811-5ab6-4174-8c98-bc9c5770aa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:19.817985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:19.832310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 755/1508] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:50:19.843332 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:19.844601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:19.951934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173524c-3c49-4172-812e-eee4e331502e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:20.015202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:20.036747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d69d77-ec26-4a15-afe7-ef64bec0f55d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:20.037405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69d77-ec26-4a15-afe7-ef64bec0f55d service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:20.037688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69d77-ec26-4a15-afe7-ef64bec0f55d service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:20.037965 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69d77-ec26-4a15-afe7-ef64bec0f55d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:20.039626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e1d69d77-ec26-4a15-afe7-ef64bec0f55d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:20.178116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d69d77-ec26-4a15-afe7-ef64bec0f55d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:20.180107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d69d77-ec26-4a15-afe7-ef64bec0f55d service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:20.200047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 754/1509] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:50:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:50:20.567377 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47c811-5ab6-4174-8c98-bc9c5770aa4e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:50:20.568707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6c47c811-5ab6-4174-8c98-bc9c5770aa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:50:20.581528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47c811-5ab6-4174-8c98-bc9c5770aa4e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:50:20.684803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:50:20.685663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:50:20.685948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:50:20.686230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] PATH_INFO: `/v3/projects/d6f5d35ce2b74281984ba9b20ddcc968` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:50:20.699092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f5d35ce2b74281984ba9b20ddcc968)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:50:20.700133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: WARNING py.warnings [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:50:20.700133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: warnings.warn(msg) Jul 19 20:50:20.700133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: Jul 19 20:50:20.701366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:50:20.762900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f5d35ce2b74281984ba9b20ddcc968', 'request_id': 'req-04579669-b9fe-4ec4-a0fd-56837ba326de'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:20.772145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f5d35ce2b74281984ba9b20ddcc968', 'request_id': 'req-04579669-b9fe-4ec4-a0fd-56837ba326de'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:20.785572 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f5d35ce2b74281984ba9b20ddcc968', 'request_id': 'req-04579669-b9fe-4ec4-a0fd-56837ba326de'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:20.808491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] The token cache is being invalidate because project d6f5d35ce2b74281984ba9b20ddcc968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:50:20.808784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:20.809369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-04579669-b9fe-4ec4-a0fd-56837ba326de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:50:20.810688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 756/1510] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:50:20 2022] DELETE /identity/v3/projects/d6f5d35ce2b74281984ba9b20ddcc968 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:50:30.197730 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=62 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:50:30.806414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:52:09.087050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3339c-dd60-407c-b958-f9b003fc14ce None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:09.087782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3339c-dd60-407c-b958-f9b003fc14ce None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:09.087782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3339c-dd60-407c-b958-f9b003fc14ce None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:09.089615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:52:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:52:09.095740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5e4cc-a8ac-4f8f-9aea-439b8dcb86a4 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:09.096403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5e4cc-a8ac-4f8f-9aea-439b8dcb86a4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:09.096403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5e4cc-a8ac-4f8f-9aea-439b8dcb86a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:09.209653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-a4f5e4cc-a8ac-4f8f-9aea-439b8dcb86a4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:09.210926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a4f5e4cc-a8ac-4f8f-9aea-439b8dcb86a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:09.353282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:09.526674 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3339c-dd60-407c-b958-f9b003fc14ce None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:09.529140 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-afe3339c-dd60-407c-b958-f9b003fc14ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:09.678869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3339c-dd60-407c-b958-f9b003fc14ce None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:09.688627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdece6f-740c-4314-990c-e886cd598651 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:09.689713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdece6f-740c-4314-990c-e886cd598651 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:09.690174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdece6f-740c-4314-990c-e886cd598651 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:09.690664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdece6f-740c-4314-990c-e886cd598651 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:09.705663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdece6f-740c-4314-990c-e886cd598651 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:09.709176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdece6f-740c-4314-990c-e886cd598651 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:09.725553 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 756/1513] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:52:10.351041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91fab4-6849-47c1-852c-cadf2d19e9e1 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:10.351877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91fab4-6849-47c1-852c-cadf2d19e9e1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:10.351877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91fab4-6849-47c1-852c-cadf2d19e9e1 None None] PATH_INFO: `/v3` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:10.353991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 758/1514] 10.222.0.94 () {54 vars in 925 bytes} [Tue Jul 19 20:52:10 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:52:10.362012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fdee0-efee-4c25-94ee-c2c42f91a88b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:10.363198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fdee0-efee-4c25-94ee-c2c42f91a88b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:10.363198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fdee0-efee-4c25-94ee-c2c42f91a88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:10.447397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e20fdee0-efee-4c25-94ee-c2c42f91a88b None None] MFA Rules not processed for user `fffebfc51dba47f68263c0bcd6b18d22`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:10.448716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e20fdee0-efee-4c25-94ee-c2c42f91a88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:10.564995 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:52:10.575971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 757/1515] 10.222.0.94 () {58 vars in 1015 bytes} [Tue Jul 19 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:10.750769 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef489085-a3a8-4794-82b3-6b20765733de None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:10.751191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef489085-a3a8-4794-82b3-6b20765733de None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:10.751589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef489085-a3a8-4794-82b3-6b20765733de None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:10.753295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 759/1516] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:52:10.759518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e58ef-b566-44a2-a01c-9916392903c5 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:10.759806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e58ef-b566-44a2-a01c-9916392903c5 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:10.760084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e58ef-b566-44a2-a01c-9916392903c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:10.782428 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-972e58ef-b566-44a2-a01c-9916392903c5 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:10.784526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-972e58ef-b566-44a2-a01c-9916392903c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:10.795373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 758/1517] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:10.976985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef489085-a3a8-4794-82b3-6b20765733de None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:10.993660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef489085-a3a8-4794-82b3-6b20765733de None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:10.999839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a3534-5858-45c0-95ad-c30b2c59e16d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:11.000611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a3534-5858-45c0-95ad-c30b2c59e16d service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:11.000926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a3534-5858-45c0-95ad-c30b2c59e16d service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:11.001207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a3534-5858-45c0-95ad-c30b2c59e16d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:11.010971 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a3534-5858-45c0-95ad-c30b2c59e16d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:11.012700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a3534-5858-45c0-95ad-c30b2c59e16d service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:11.024439 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 760/1518] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:52:20.710572 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d557e4-589e-4783-987d-e8a9671f0637 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:20.711131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d557e4-589e-4783-987d-e8a9671f0637 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:20.711131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d557e4-589e-4783-987d-e8a9671f0637 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:20.712551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 759/1519] 10.222.0.85 () {54 vars in 961 bytes} [Tue Jul 19 20:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:52:20.718015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1381f5-2351-4e4c-99a8-2addbcb1f290 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:20.718581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1381f5-2351-4e4c-99a8-2addbcb1f290 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:20.718581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1381f5-2351-4e4c-99a8-2addbcb1f290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:20.737963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-7b1381f5-2351-4e4c-99a8-2addbcb1f290 None None] MFA Rules not processed for user `fffebfc51dba47f68263c0bcd6b18d22`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:20.739726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7b1381f5-2351-4e4c-99a8-2addbcb1f290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:20.748911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 761/1520] 10.222.0.85 () {58 vars in 1058 bytes} [Tue Jul 19 20:52:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:20.757276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d557e4-589e-4783-987d-e8a9671f0637 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:20.771225 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d557e4-589e-4783-987d-e8a9671f0637 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:20.776567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c37d6-3c5b-45cb-92bd-4578c2390f0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:20.777250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c37d6-3c5b-45cb-92bd-4578c2390f0d service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:20.777510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c37d6-3c5b-45cb-92bd-4578c2390f0d service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:20.777782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c37d6-3c5b-45cb-92bd-4578c2390f0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:20.787153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c37d6-3c5b-45cb-92bd-4578c2390f0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:20.788804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c37d6-3c5b-45cb-92bd-4578c2390f0d service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:20.799769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 760/1521] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:52:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:52:22.351530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a9c8-fa89-4e54-badb-d6f0b48fea1c None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:22.352235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a9c8-fa89-4e54-badb-d6f0b48fea1c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:22.352235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a9c8-fa89-4e54-badb-d6f0b48fea1c None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:22.353521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:52:22.360769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6a144-fc2c-43be-86da-8c99489ff3f8 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:22.361226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6a144-fc2c-43be-86da-8c99489ff3f8 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:22.361338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6a144-fc2c-43be-86da-8c99489ff3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:22.380025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d4e6a144-fc2c-43be-86da-8c99489ff3f8 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:22.381203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d4e6a144-fc2c-43be-86da-8c99489ff3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:22.389191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:22.543268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718a9c8-fa89-4e54-badb-d6f0b48fea1c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:22.558633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718a9c8-fa89-4e54-badb-d6f0b48fea1c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:22.564352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac001e47-21ca-4091-b8a8-e0a016ae91cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:22.565001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac001e47-21ca-4091-b8a8-e0a016ae91cb service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:22.565286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac001e47-21ca-4091-b8a8-e0a016ae91cb service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:22.565560 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac001e47-21ca-4091-b8a8-e0a016ae91cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:22.574470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac001e47-21ca-4091-b8a8-e0a016ae91cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:22.576014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac001e47-21ca-4091-b8a8-e0a016ae91cb service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:22.596258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 763/1524] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:52:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:52:23.470035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74d744-9aa7-464f-b5dc-51194bb0fbca None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:23.470604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74d744-9aa7-464f-b5dc-51194bb0fbca None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:23.470604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74d744-9aa7-464f-b5dc-51194bb0fbca None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:23.471859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 20:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:52:23.477790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb4da9-4215-45e4-b12e-1cb50eb39cdb None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:23.478538 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb4da9-4215-45e4-b12e-1cb50eb39cdb None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:23.478538 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb4da9-4215-45e4-b12e-1cb50eb39cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:23.509689 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-67eb4da9-4215-45e4-b12e-1cb50eb39cdb None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:23.510913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-67eb4da9-4215-45e4-b12e-1cb50eb39cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:23.520947 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:23.682375 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74d744-9aa7-464f-b5dc-51194bb0fbca None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:23.697339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74d744-9aa7-464f-b5dc-51194bb0fbca None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:23.702886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f02e4a-28bb-4e53-bec9-af0f9909cf75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:23.703522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f02e4a-28bb-4e53-bec9-af0f9909cf75 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:23.703816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f02e4a-28bb-4e53-bec9-af0f9909cf75 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:23.704086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f02e4a-28bb-4e53-bec9-af0f9909cf75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:23.712864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f02e4a-28bb-4e53-bec9-af0f9909cf75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:23.714476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f02e4a-28bb-4e53-bec9-af0f9909cf75 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:23.724738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 763/1527] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:52:27.867518 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368935c8-d6a1-4aa4-b2e2-c61ea593b276 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:27.868071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368935c8-d6a1-4aa4-b2e2-c61ea593b276 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:27.868071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368935c8-d6a1-4aa4-b2e2-c61ea593b276 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:27.869424 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:52:27.874863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1ba87-4d48-4daa-add1-7e42254eeb30 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:27.875457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1ba87-4d48-4daa-add1-7e42254eeb30 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:27.875457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1ba87-4d48-4daa-add1-7e42254eeb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:27.899774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-fcf1ba87-4d48-4daa-add1-7e42254eeb30 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:27.901016 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fcf1ba87-4d48-4daa-add1-7e42254eeb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:27.910476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:27.996044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368935c8-d6a1-4aa4-b2e2-c61ea593b276 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:27.997824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-368935c8-d6a1-4aa4-b2e2-c61ea593b276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:28.035065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:52:28.092443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368935c8-d6a1-4aa4-b2e2-c61ea593b276 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:28.097312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b286202-e941-4dfe-bffa-e231e3e42222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:28.097978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b286202-e941-4dfe-bffa-e231e3e42222 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:28.098306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b286202-e941-4dfe-bffa-e231e3e42222 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:28.098543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b286202-e941-4dfe-bffa-e231e3e42222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:28.110095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b286202-e941-4dfe-bffa-e231e3e42222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:28.111523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b286202-e941-4dfe-bffa-e231e3e42222 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:28.126376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 766/1530] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:52:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:52:30.301379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f790d-b4db-4d7d-a75e-a7cf54830498 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:30.302174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f790d-b4db-4d7d-a75e-a7cf54830498 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:30.302174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f790d-b4db-4d7d-a75e-a7cf54830498 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:30.303288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:52:30.308911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25553e3-8b4b-4613-887a-890cde7d0d45 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:30.309416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25553e3-8b4b-4613-887a-890cde7d0d45 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:30.309512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25553e3-8b4b-4613-887a-890cde7d0d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:30.328950 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-b25553e3-8b4b-4613-887a-890cde7d0d45 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:30.330165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b25553e3-8b4b-4613-887a-890cde7d0d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:30.339035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:30.377044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f790d-b4db-4d7d-a75e-a7cf54830498 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:30.390716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f790d-b4db-4d7d-a75e-a7cf54830498 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:30.396320 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63fcd1-f648-4b6d-8871-5728537f3eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:30.397267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63fcd1-f648-4b6d-8871-5728537f3eaf service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:30.397672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63fcd1-f648-4b6d-8871-5728537f3eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:30.398068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63fcd1-f648-4b6d-8871-5728537f3eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:30.411355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63fcd1-f648-4b6d-8871-5728537f3eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:30.413024 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63fcd1-f648-4b6d-8871-5728537f3eaf service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:30.423606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 766/1533] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:52:34.262345 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331cf96-b285-45c3-9b8f-0b4cd10db5ba None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:34.263199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331cf96-b285-45c3-9b8f-0b4cd10db5ba None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:34.263199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331cf96-b285-45c3-9b8f-0b4cd10db5ba None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:34.264842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:52:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 20:52:34.271622 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59a073-c859-4dd5-a1ce-99ec39d99217 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:34.272283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59a073-c859-4dd5-a1ce-99ec39d99217 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:34.272283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59a073-c859-4dd5-a1ce-99ec39d99217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:34.293883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-da59a073-c859-4dd5-a1ce-99ec39d99217 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:34.295118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-da59a073-c859-4dd5-a1ce-99ec39d99217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:34.304204 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:34.624352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331cf96-b285-45c3-9b8f-0b4cd10db5ba None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:34.626281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8331cf96-b285-45c3-9b8f-0b4cd10db5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:34.716660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331cf96-b285-45c3-9b8f-0b4cd10db5ba None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:34.720652 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d187345-9973-409d-83e7-c6dd2ce42d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:34.721390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d187345-9973-409d-83e7-c6dd2ce42d16 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:34.721843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d187345-9973-409d-83e7-c6dd2ce42d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:34.722140 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d187345-9973-409d-83e7-c6dd2ce42d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:34.736163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d187345-9973-409d-83e7-c6dd2ce42d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:34.739829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d187345-9973-409d-83e7-c6dd2ce42d16 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:34.753776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 769/1536] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:52:36.862239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b8c26-f7fb-4b2c-8bf1-e52da290f17f None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:36.862837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b8c26-f7fb-4b2c-8bf1-e52da290f17f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:36.862837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b8c26-f7fb-4b2c-8bf1-e52da290f17f None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:36.864791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 20:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:52:36.870188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e1226-108b-4cd0-a57a-f893e887aa5e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:36.870934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e1226-108b-4cd0-a57a-f893e887aa5e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:36.870934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e1226-108b-4cd0-a57a-f893e887aa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:36.890250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-098e1226-108b-4cd0-a57a-f893e887aa5e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:52:36.891480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-098e1226-108b-4cd0-a57a-f893e887aa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:36.899593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:52:37.210741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b8c26-f7fb-4b2c-8bf1-e52da290f17f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:52:37.212664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-025b8c26-f7fb-4b2c-8bf1-e52da290f17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:52:37.226376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b8c26-f7fb-4b2c-8bf1-e52da290f17f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:52:37.230209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792260b-a12d-47b0-a759-ba030be3eecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:52:37.231070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792260b-a12d-47b0-a759-ba030be3eecb service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:52:37.231209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792260b-a12d-47b0-a759-ba030be3eecb service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:52:37.231409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792260b-a12d-47b0-a759-ba030be3eecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:52:37.242124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792260b-a12d-47b0-a759-ba030be3eecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:52:37.243565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792260b-a12d-47b0-a759-ba030be3eecb service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:52:37.255264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 769/1539] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:52:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:52:46.884970 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=63 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:52:47.249305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=37 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:13.370264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7cbe2-28f8-4222-80e7-cde211eac6b5 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:13.371440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7cbe2-28f8-4222-80e7-cde211eac6b5 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:13.371440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7cbe2-28f8-4222-80e7-cde211eac6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:13.398490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-25c7cbe2-28f8-4222-80e7-cde211eac6b5 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:13.399993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-25c7cbe2-28f8-4222-80e7-cde211eac6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:13.416470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 771/1540] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:13.436747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417f627-235c-4466-89f0-8c26dac7da95 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:13.437716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417f627-235c-4466-89f0-8c26dac7da95 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:13.437716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417f627-235c-4466-89f0-8c26dac7da95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:13.469639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c417f627-235c-4466-89f0-8c26dac7da95 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:13.471474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c417f627-235c-4466-89f0-8c26dac7da95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:13.484317 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 770/1541] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:54:13.490372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7cbe2-28f8-4222-80e7-cde211eac6b5 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:13.503952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7cbe2-28f8-4222-80e7-cde211eac6b5 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:13.510343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84c717-584a-4559-a55c-7545b2764522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:13.511525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84c717-584a-4559-a55c-7545b2764522 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:13.511825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84c717-584a-4559-a55c-7545b2764522 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:13.512110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84c717-584a-4559-a55c-7545b2764522 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:13.512942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84c717-584a-4559-a55c-7545b2764522 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:13.513260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84c717-584a-4559-a55c-7545b2764522 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:13.514973 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84c717-584a-4559-a55c-7545b2764522 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:13.533928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 772/1542] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:13.543357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417f627-235c-4466-89f0-8c26dac7da95 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:13.556715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417f627-235c-4466-89f0-8c26dac7da95 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:13.563732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875287b2-65a3-47cd-9400-eca7697a6534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:13.564982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875287b2-65a3-47cd-9400-eca7697a6534 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:13.565266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875287b2-65a3-47cd-9400-eca7697a6534 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:13.565541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875287b2-65a3-47cd-9400-eca7697a6534 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:13.566260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875287b2-65a3-47cd-9400-eca7697a6534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:13.568490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875287b2-65a3-47cd-9400-eca7697a6534 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:13.627354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 771/1543] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:13 2022] POST /identity/v3/projects => generated 365 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:13.634017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84c717-584a-4559-a55c-7545b2764522 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:13.654092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84c717-584a-4559-a55c-7545b2764522 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:13.785485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e0b92-394a-49da-a63c-8c9cde03b209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:13.786201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e0b92-394a-49da-a63c-8c9cde03b209 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:13.786673 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e0b92-394a-49da-a63c-8c9cde03b209 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:13.787415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e0b92-394a-49da-a63c-8c9cde03b209 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:13.788781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e0b92-394a-49da-a63c-8c9cde03b209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:13.792901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e0b92-394a-49da-a63c-8c9cde03b209 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:13.834359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 773/1544] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:13 2022] POST /identity/v3/users => generated 283 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:13.841138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875287b2-65a3-47cd-9400-eca7697a6534 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:13.863518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875287b2-65a3-47cd-9400-eca7697a6534 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:13.872388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2457e-6f2d-4301-87fc-f10dcd4556ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:13.873511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2457e-6f2d-4301-87fc-f10dcd4556ff admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:13.874033 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2457e-6f2d-4301-87fc-f10dcd4556ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:13.874554 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2457e-6f2d-4301-87fc-f10dcd4556ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:13.875633 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2457e-6f2d-4301-87fc-f10dcd4556ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:13.877830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2457e-6f2d-4301-87fc-f10dcd4556ff admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:13.884455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3011b-3eb9-4f11-9ce6-039f102080f4 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:13.885106 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3011b-3eb9-4f11-9ce6-039f102080f4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:13.885593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3011b-3eb9-4f11-9ce6-039f102080f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:13.897046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 772/1545] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:54:13.903898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2457e-6f2d-4301-87fc-f10dcd4556ff admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:13.922485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2457e-6f2d-4301-87fc-f10dcd4556ff admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:13.936695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81db5d1-4f3f-4f09-a473-6030f6d69e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:13.937782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81db5d1-4f3f-4f09-a473-6030f6d69e7a admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:13.938574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-90e3011b-3eb9-4f11-9ce6-039f102080f4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:13.938792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81db5d1-4f3f-4f09-a473-6030f6d69e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:13.939269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81db5d1-4f3f-4f09-a473-6030f6d69e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:13.940168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81db5d1-4f3f-4f09-a473-6030f6d69e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:13.941920 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-90e3011b-3eb9-4f11-9ce6-039f102080f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:13.942206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81db5d1-4f3f-4f09-a473-6030f6d69e7a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:13.961232 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 773/1546] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:54:13.963368 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a975beb-247d-4ff4-b0d3-3e71c62451ec None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:13.963989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a975beb-247d-4ff4-b0d3-3e71c62451ec None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:13.964560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a975beb-247d-4ff4-b0d3-3e71c62451ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:13.980382 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 774/1547] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:13.982510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3011b-3eb9-4f11-9ce6-039f102080f4 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:14.001310 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4a975beb-247d-4ff4-b0d3-3e71c62451ec None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.002872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4a975beb-247d-4ff4-b0d3-3e71c62451ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.007827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3011b-3eb9-4f11-9ce6-039f102080f4 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:14.021088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 774/1548] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.021646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fffc0-e9f4-4a50-9b85-2389ff022c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:14.023352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fffc0-e9f4-4a50-9b85-2389ff022c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.023866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fffc0-e9f4-4a50-9b85-2389ff022c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.024161 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def72a9-9347-4323-8d5c-eb4f3351e2d3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.024386 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fffc0-e9f4-4a50-9b85-2389ff022c9c admin admin] PATH_INFO: `/v3/projects/46aced1136734ec4b31804fb64155e88/users/000d34f0bcd1443fa703cd68bded1c5e/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.024980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def72a9-9347-4323-8d5c-eb4f3351e2d3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.025639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def72a9-9347-4323-8d5c-eb4f3351e2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.077462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fffc0-e9f4-4a50-9b85-2389ff022c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46aced1136734ec4b31804fb64155e88, user_id=000d34f0bcd1443fa703cd68bded1c5e, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:14.079635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: WARNING py.warnings [None req-f97fffc0-e9f4-4a50-9b85-2389ff022c9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:54:14.079635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: warnings.warn(msg) Jul 19 20:54:14.079635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: Jul 19 20:54:14.085906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3def72a9-9347-4323-8d5c-eb4f3351e2d3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.087449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fffc0-e9f4-4a50-9b85-2389ff022c9c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:14.089346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3def72a9-9347-4323-8d5c-eb4f3351e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.110406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 775/1549] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.115766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5d80-deeb-414b-8cc1-6e13549786d0 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.116500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5d80-deeb-414b-8cc1-6e13549786d0 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.117598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 775/1550] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:13 2022] PUT /identity/v3/projects/46aced1136734ec4b31804fb64155e88/users/000d34f0bcd1443fa703cd68bded1c5e/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:14.118348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5d80-deeb-414b-8cc1-6e13549786d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.122975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c81c5-c394-4328-a6ca-84ae32de8fb4 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.123675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c81c5-c394-4328-a6ca-84ae32de8fb4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.125019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c81c5-c394-4328-a6ca-84ae32de8fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.154995 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-aa6a5d80-deeb-414b-8cc1-6e13549786d0 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.157087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-aa6a5d80-deeb-414b-8cc1-6e13549786d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.164901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-576c81c5-c394-4328-a6ca-84ae32de8fb4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.167342 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-576c81c5-c394-4328-a6ca-84ae32de8fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.281663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 776/1551] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.284799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b8bd-aeba-43f0-b262-54f5fbde8860 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.285331 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b8bd-aeba-43f0-b262-54f5fbde8860 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.285845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b8bd-aeba-43f0-b262-54f5fbde8860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.303917 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 776/1552] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.305785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c81c5-c394-4328-a6ca-84ae32de8fb4 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:14.319941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c81c5-c394-4328-a6ca-84ae32de8fb4 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:14.322360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-efe4b8bd-aeba-43f0-b262-54f5fbde8860 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.323547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-efe4b8bd-aeba-43f0-b262-54f5fbde8860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.327709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e08fe1-0725-4ff0-ac7b-f7c99269b8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:14.328814 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e08fe1-0725-4ff0-ac7b-f7c99269b8fd admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.329866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e08fe1-0725-4ff0-ac7b-f7c99269b8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.330387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e08fe1-0725-4ff0-ac7b-f7c99269b8fd admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.331425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e08fe1-0725-4ff0-ac7b-f7c99269b8fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:14.332031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e08fe1-0725-4ff0-ac7b-f7c99269b8fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:14.333770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 777/1553] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.335026 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e08fe1-0725-4ff0-ac7b-f7c99269b8fd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:14.335814 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c00990-dd3d-43ff-84f1-6958330e087e None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.336298 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c00990-dd3d-43ff-84f1-6958330e087e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.339363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c00990-dd3d-43ff-84f1-6958330e087e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.355769 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 777/1554] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:14.357686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f112c05-2bd7-495e-8aee-6c8fd4b1baa5 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.358128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f112c05-2bd7-495e-8aee-6c8fd4b1baa5 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.358617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f112c05-2bd7-495e-8aee-6c8fd4b1baa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.384137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-f7c00990-dd3d-43ff-84f1-6958330e087e None None] MFA Rules not processed for user `000d34f0bcd1443fa703cd68bded1c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.385461 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f7c00990-dd3d-43ff-84f1-6958330e087e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.386674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-5f112c05-2bd7-495e-8aee-6c8fd4b1baa5 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.388699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-5f112c05-2bd7-495e-8aee-6c8fd4b1baa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.402924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 778/1555] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.405459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f551313-2929-4761-beda-7e0482f1831e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.405958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f551313-2929-4761-beda-7e0482f1831e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.406892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f551313-2929-4761-beda-7e0482f1831e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.440026 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-4f551313-2929-4761-beda-7e0482f1831e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.442200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4f551313-2929-4761-beda-7e0482f1831e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.457624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 779/1556] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.460069 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec885723-63ac-45a6-ae7b-f639eff2d21e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.460399 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec885723-63ac-45a6-ae7b-f639eff2d21e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.460934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec885723-63ac-45a6-ae7b-f639eff2d21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.493393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 778/1557] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.494357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c00990-dd3d-43ff-84f1-6958330e087e None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:14.501253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ec885723-63ac-45a6-ae7b-f639eff2d21e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.503770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ec885723-63ac-45a6-ae7b-f639eff2d21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.506429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c00990-dd3d-43ff-84f1-6958330e087e None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:14.511805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5d619-7e33-4730-b578-00669bb2084a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:14.512508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5d619-7e33-4730-b578-00669bb2084a admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.512800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5d619-7e33-4730-b578-00669bb2084a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.513068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5d619-7e33-4730-b578-00669bb2084a admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.513778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5d619-7e33-4730-b578-00669bb2084a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:14.514079 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5d619-7e33-4730-b578-00669bb2084a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:14.515112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: WARNING py.warnings [None req-8dd5d619-7e33-4730-b578-00669bb2084a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:54:14.515112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: warnings.warn(msg) Jul 19 20:54:14.515112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: Jul 19 20:54:14.516007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5d619-7e33-4730-b578-00669bb2084a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:14.518848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 780/1558] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.522839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a9927-82c3-4966-8d14-846bc522f33d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.523543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a9927-82c3-4966-8d14-846bc522f33d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.524017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a9927-82c3-4966-8d14-846bc522f33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.525347 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:14.533830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 779/1559] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:14.535361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0807811-9915-4ab8-a65d-ca1773bed25b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.535557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0807811-9915-4ab8-a65d-ca1773bed25b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.535825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0807811-9915-4ab8-a65d-ca1773bed25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.547820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:14.557368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-523a9927-82c3-4966-8d14-846bc522f33d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.558043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e0807811-9915-4ab8-a65d-ca1773bed25b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.559365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e0807811-9915-4ab8-a65d-ca1773bed25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.560133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-523a9927-82c3-4966-8d14-846bc522f33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.569061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 780/1560] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.569692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0807811-9915-4ab8-a65d-ca1773bed25b None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:14.573306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 781/1561] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.576031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d3fe-5600-4c49-9b4b-7eaa6d102626 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.576568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d3fe-5600-4c49-9b4b-7eaa6d102626 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.577068 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d3fe-5600-4c49-9b4b-7eaa6d102626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.585376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0807811-9915-4ab8-a65d-ca1773bed25b None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:14.594701 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594fc98-77cd-4e82-a767-8a1a9c4431bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:14.596031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594fc98-77cd-4e82-a767-8a1a9c4431bb admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.596547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594fc98-77cd-4e82-a767-8a1a9c4431bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.597018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594fc98-77cd-4e82-a767-8a1a9c4431bb admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.600099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594fc98-77cd-4e82-a767-8a1a9c4431bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:14.602203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594fc98-77cd-4e82-a767-8a1a9c4431bb admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:14.608366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-3c88d3fe-5600-4c49-9b4b-7eaa6d102626 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.610348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3c88d3fe-5600-4c49-9b4b-7eaa6d102626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.629088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 782/1562] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.629088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25711311-dcf2-4b46-bc25-3382d511f5ee None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.629088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25711311-dcf2-4b46-bc25-3382d511f5ee None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.629088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25711311-dcf2-4b46-bc25-3382d511f5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.637935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 781/1563] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:14.639114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594fc98-77cd-4e82-a767-8a1a9c4431bb admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:14.663199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-25711311-dcf2-4b46-bc25-3382d511f5ee None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.665262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-25711311-dcf2-4b46-bc25-3382d511f5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.708482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594fc98-77cd-4e82-a767-8a1a9c4431bb admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:14.714879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b653ec-6c09-436d-b0f4-c6b517fed9f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:14.715770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b653ec-6c09-436d-b0f4-c6b517fed9f8 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.716107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b653ec-6c09-436d-b0f4-c6b517fed9f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.716486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b653ec-6c09-436d-b0f4-c6b517fed9f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.782337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 783/1564] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.785372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2373b6-b6d5-4e3a-86a4-323c35bb9065 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.785372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2373b6-b6d5-4e3a-86a4-323c35bb9065 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.785716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2373b6-b6d5-4e3a-86a4-323c35bb9065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.809200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ca2373b6-b6d5-4e3a-86a4-323c35bb9065 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:14.810499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ca2373b6-b6d5-4e3a-86a4-323c35bb9065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:14.819656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b653ec-6c09-436d-b0f4-c6b517fed9f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:14.821580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b653ec-6c09-436d-b0f4-c6b517fed9f8 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:14.827868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 784/1565] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:14.827868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2373b6-b6d5-4e3a-86a4-323c35bb9065 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:14.839982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 782/1566] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:14.840269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2373b6-b6d5-4e3a-86a4-323c35bb9065 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:14.840676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b653ec-6c09-436d-b0f4-c6b517fed9f8 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:14.846868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7464e-1d55-485b-985a-7a433fd0ff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:14.848710 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7464e-1d55-485b-985a-7a433fd0ff24 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.849186 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7464e-1d55-485b-985a-7a433fd0ff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.850047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7464e-1d55-485b-985a-7a433fd0ff24 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.851149 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7464e-1d55-485b-985a-7a433fd0ff24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:14.852827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: WARNING py.warnings [None req-c5e7464e-1d55-485b-985a-7a433fd0ff24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:54:14.852827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: warnings.warn(msg) Jul 19 20:54:14.852827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: Jul 19 20:54:14.853780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b653ec-6c09-436d-b0f4-c6b517fed9f8 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:14.854692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7464e-1d55-485b-985a-7a433fd0ff24 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:14.863289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b20108-b7d7-456d-84cd-02d71eb955e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:14.864490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b20108-b7d7-456d-84cd-02d71eb955e8 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:14.865015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b20108-b7d7-456d-84cd-02d71eb955e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:14.865540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b20108-b7d7-456d-84cd-02d71eb955e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:14.866921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b20108-b7d7-456d-84cd-02d71eb955e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:14.867468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b20108-b7d7-456d-84cd-02d71eb955e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:14.870595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b20108-b7d7-456d-84cd-02d71eb955e8 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:14.897681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 783/1567] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:14.897957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 785/1568] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:14 2022] POST /identity/v3/projects => generated 369 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:14.897957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7464e-1d55-485b-985a-7a433fd0ff24 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:14.898119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b20108-b7d7-456d-84cd-02d71eb955e8 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:14.914733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b20108-b7d7-456d-84cd-02d71eb955e8 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.003705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7464e-1d55-485b-985a-7a433fd0ff24 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.009458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a528ee1-b84c-4777-99b3-cbf3cdfec925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.010138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a528ee1-b84c-4777-99b3-cbf3cdfec925 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.010527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a528ee1-b84c-4777-99b3-cbf3cdfec925 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.010890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a528ee1-b84c-4777-99b3-cbf3cdfec925 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.012142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a528ee1-b84c-4777-99b3-cbf3cdfec925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:15.012524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a528ee1-b84c-4777-99b3-cbf3cdfec925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.014235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a528ee1-b84c-4777-99b3-cbf3cdfec925 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.027516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe889c53-38e3-4ce0-bca3-04480c9c19f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.028687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe889c53-38e3-4ce0-bca3-04480c9c19f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.029158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 786/1569] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:15.029675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe889c53-38e3-4ce0-bca3-04480c9c19f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.030014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a528ee1-b84c-4777-99b3-cbf3cdfec925 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.030414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe889c53-38e3-4ce0-bca3-04480c9c19f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.031400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe889c53-38e3-4ce0-bca3-04480c9c19f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:15.031714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe889c53-38e3-4ce0-bca3-04480c9c19f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.037137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe889c53-38e3-4ce0-bca3-04480c9c19f0 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.064472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 784/1570] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:15.064703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a528ee1-b84c-4777-99b3-cbf3cdfec925 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.064893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe889c53-38e3-4ce0-bca3-04480c9c19f0 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.072780 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d5107-95c6-42c9-adc4-e52457d8cbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.074326 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d5107-95c6-42c9-adc4-e52457d8cbf4 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.074485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:15.074579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d5107-95c6-42c9-adc4-e52457d8cbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.075078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d5107-95c6-42c9-adc4-e52457d8cbf4 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.076738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d5107-95c6-42c9-adc4-e52457d8cbf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.080666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d5107-95c6-42c9-adc4-e52457d8cbf4 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.081826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe889c53-38e3-4ce0-bca3-04480c9c19f0 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.089898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872604e-6990-4cad-90c3-7cc1d6c17737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.091027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872604e-6990-4cad-90c3-7cc1d6c17737 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.091425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872604e-6990-4cad-90c3-7cc1d6c17737 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.091896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872604e-6990-4cad-90c3-7cc1d6c17737 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.093188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872604e-6990-4cad-90c3-7cc1d6c17737 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:15.093674 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872604e-6990-4cad-90c3-7cc1d6c17737 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.097544 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872604e-6990-4cad-90c3-7cc1d6c17737 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.120965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 787/1571] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:15 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:15.121936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d5107-95c6-42c9-adc4-e52457d8cbf4 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.127132 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 785/1572] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:15.128660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872604e-6990-4cad-90c3-7cc1d6c17737 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.135457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d5107-95c6-42c9-adc4-e52457d8cbf4 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.143208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72551b6c-0b39-4584-8580-f1a23d3315bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.143999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72551b6c-0b39-4584-8580-f1a23d3315bc admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.144294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72551b6c-0b39-4584-8580-f1a23d3315bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.144727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72551b6c-0b39-4584-8580-f1a23d3315bc admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.145524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72551b6c-0b39-4584-8580-f1a23d3315bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:15.145903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72551b6c-0b39-4584-8580-f1a23d3315bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.146129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872604e-6990-4cad-90c3-7cc1d6c17737 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.147576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72551b6c-0b39-4584-8580-f1a23d3315bc admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.157628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975167a-f720-4050-b17d-8125952be1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.160071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975167a-f720-4050-b17d-8125952be1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.160477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975167a-f720-4050-b17d-8125952be1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.160785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975167a-f720-4050-b17d-8125952be1d3 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.161752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975167a-f720-4050-b17d-8125952be1d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.163072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: WARNING py.warnings [None req-d975167a-f720-4050-b17d-8125952be1d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:54:15.163072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: warnings.warn(msg) Jul 19 20:54:15.163072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: Jul 19 20:54:15.164407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975167a-f720-4050-b17d-8125952be1d3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.170722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 788/1573] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:15.172619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72551b6c-0b39-4584-8580-f1a23d3315bc admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.194488 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72551b6c-0b39-4584-8580-f1a23d3315bc admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.199811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e26bc8-4c04-4371-a736-f2179ec47be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.200532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26bc8-4c04-4371-a736-f2179ec47be5 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.200827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26bc8-4c04-4371-a736-f2179ec47be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.201129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e26bc8-4c04-4371-a736-f2179ec47be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.205609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e26bc8-4c04-4371-a736-f2179ec47be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.205884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 786/1574] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:15 2022] POST /identity/v3/users => generated 285 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:15.205984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e26bc8-4c04-4371-a736-f2179ec47be5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.210722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975167a-f720-4050-b17d-8125952be1d3 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.235387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975167a-f720-4050-b17d-8125952be1d3 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.252427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 789/1575] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:15 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:15.253053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a98725-73cc-4b11-9605-d34008ceda05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.253228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e26bc8-4c04-4371-a736-f2179ec47be5 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.253429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98725-73cc-4b11-9605-d34008ceda05 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.253429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98725-73cc-4b11-9605-d34008ceda05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.253429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98725-73cc-4b11-9605-d34008ceda05 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.253429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a98725-73cc-4b11-9605-d34008ceda05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.253976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a98725-73cc-4b11-9605-d34008ceda05 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.273786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:15.322180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 787/1576] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:15 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:15.322180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a98725-73cc-4b11-9605-d34008ceda05 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.376762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e26bc8-4c04-4371-a736-f2179ec47be5 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.433458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a98725-73cc-4b11-9605-d34008ceda05 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.443190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5fbb2-fc5e-4a43-becc-bdfec749998a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.444182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5fbb2-fc5e-4a43-becc-bdfec749998a admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.444661 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5fbb2-fc5e-4a43-becc-bdfec749998a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.445116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5fbb2-fc5e-4a43-becc-bdfec749998a admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.446830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5fbb2-fc5e-4a43-becc-bdfec749998a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.449764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5fbb2-fc5e-4a43-becc-bdfec749998a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.472349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 788/1577] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:15.472349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5fbb2-fc5e-4a43-becc-bdfec749998a admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.480010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5137d-d65a-40da-99aa-a14b85687186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.481083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5137d-d65a-40da-99aa-a14b85687186 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.481503 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5137d-d65a-40da-99aa-a14b85687186 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.481916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5137d-d65a-40da-99aa-a14b85687186 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.483071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5137d-d65a-40da-99aa-a14b85687186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.486532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5137d-d65a-40da-99aa-a14b85687186 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.488834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5fbb2-fc5e-4a43-becc-bdfec749998a admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.497417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c9b18-ce30-495e-9e7d-ca0620c9f657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.498478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c9b18-ce30-495e-9e7d-ca0620c9f657 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.498938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c9b18-ce30-495e-9e7d-ca0620c9f657 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.499415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c9b18-ce30-495e-9e7d-ca0620c9f657 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.500600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c9b18-ce30-495e-9e7d-ca0620c9f657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.503618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c9b18-ce30-495e-9e7d-ca0620c9f657 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.536565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 790/1578] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:15 2022] POST /identity/v3/projects => generated 371 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:15.542932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5137d-d65a-40da-99aa-a14b85687186 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.547304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 789/1579] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:15 2022] POST /identity/v3/projects => generated 387 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:15.547915 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c9b18-ce30-495e-9e7d-ca0620c9f657 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.678076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5137d-d65a-40da-99aa-a14b85687186 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.724194 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c9b18-ce30-495e-9e7d-ca0620c9f657 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.734600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a9727-b42a-40b3-9edf-e5575f614f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.735728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a9727-b42a-40b3-9edf-e5575f614f21 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.736259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a9727-b42a-40b3-9edf-e5575f614f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.736784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a9727-b42a-40b3-9edf-e5575f614f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.738308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a9727-b42a-40b3-9edf-e5575f614f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.742690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a9727-b42a-40b3-9edf-e5575f614f21 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.770907 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 790/1580] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:15.773099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a9727-b42a-40b3-9edf-e5575f614f21 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.797231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:15.798671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a9727-b42a-40b3-9edf-e5575f614f21 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.806447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d4f7c-c524-4b77-a9fb-525e9cadca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.807604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d4f7c-c524-4b77-a9fb-525e9cadca4e admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.808099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d4f7c-c524-4b77-a9fb-525e9cadca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.808671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d4f7c-c524-4b77-a9fb-525e9cadca4e admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.810267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d4f7c-c524-4b77-a9fb-525e9cadca4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.815146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d4f7c-c524-4b77-a9fb-525e9cadca4e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.818901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5feecb-ee60-4994-844b-781c3b022c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.820149 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5feecb-ee60-4994-844b-781c3b022c07 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.820731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5feecb-ee60-4994-844b-781c3b022c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.821180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5feecb-ee60-4994-844b-781c3b022c07 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.822327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5feecb-ee60-4994-844b-781c3b022c07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.824985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5feecb-ee60-4994-844b-781c3b022c07 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.857493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 791/1581] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:15 2022] POST /identity/v3/users => generated 292 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:15.859659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d4f7c-c524-4b77-a9fb-525e9cadca4e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.875525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 791/1582] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:15 2022] POST /identity/v3/projects => generated 379 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:15.875525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5feecb-ee60-4994-844b-781c3b022c07 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:15.879940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:15.888217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d4f7c-c524-4b77-a9fb-525e9cadca4e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.972544 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2687584-5be1-4890-ab0c-9b72c2c894a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:15.973315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2687584-5be1-4890-ab0c-9b72c2c894a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:15.973810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2687584-5be1-4890-ab0c-9b72c2c894a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:15.974181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2687584-5be1-4890-ab0c-9b72c2c894a3 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:15.975062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2687584-5be1-4890-ab0c-9b72c2c894a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:15.977443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2687584-5be1-4890-ab0c-9b72c2c894a3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:15.990661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5feecb-ee60-4994-844b-781c3b022c07 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:15.999613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72a84d-5434-4145-aa07-053a06f2fd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.000718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72a84d-5434-4145-aa07-053a06f2fd33 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.001385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72a84d-5434-4145-aa07-053a06f2fd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.001804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72a84d-5434-4145-aa07-053a06f2fd33 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.003408 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72a84d-5434-4145-aa07-053a06f2fd33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.006350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72a84d-5434-4145-aa07-053a06f2fd33 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.010873 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 792/1583] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:15 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:16.012315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2687584-5be1-4890-ab0c-9b72c2c894a3 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.027976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 792/1584] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:16.028372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72a84d-5434-4145-aa07-053a06f2fd33 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.029754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2687584-5be1-4890-ab0c-9b72c2c894a3 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.040679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ef09d-eae5-4dd3-9f6c-3946c47cf556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.041745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ef09d-eae5-4dd3-9f6c-3946c47cf556 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.042190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ef09d-eae5-4dd3-9f6c-3946c47cf556 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.042639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ef09d-eae5-4dd3-9f6c-3946c47cf556 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.043951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ef09d-eae5-4dd3-9f6c-3946c47cf556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.045498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72a84d-5434-4145-aa07-053a06f2fd33 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.047250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ef09d-eae5-4dd3-9f6c-3946c47cf556 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.056691 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7ee7-e2fe-4f90-8d4c-1874ccaa425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.057696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7ee7-e2fe-4f90-8d4c-1874ccaa425e admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.058211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7ee7-e2fe-4f90-8d4c-1874ccaa425e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.058659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be7ee7-e2fe-4f90-8d4c-1874ccaa425e admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.059582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be7ee7-e2fe-4f90-8d4c-1874ccaa425e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.062918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be7ee7-e2fe-4f90-8d4c-1874ccaa425e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.092214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 793/1585] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:16 2022] POST /identity/v3/users => generated 286 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:16.092214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ef09d-eae5-4dd3-9f6c-3946c47cf556 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.104499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 793/1586] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:16 2022] POST /identity/v3/users => generated 294 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:16.104499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7ee7-e2fe-4f90-8d4c-1874ccaa425e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.104964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ef09d-eae5-4dd3-9f6c-3946c47cf556 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.109775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5017aa5-7e51-4501-a713-1ecd1f3d4c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.110490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5017aa5-7e51-4501-a713-1ecd1f3d4c65 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.110780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5017aa5-7e51-4501-a713-1ecd1f3d4c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.111612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5017aa5-7e51-4501-a713-1ecd1f3d4c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.113204 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5017aa5-7e51-4501-a713-1ecd1f3d4c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.115414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be7ee7-e2fe-4f90-8d4c-1874ccaa425e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.116268 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5017aa5-7e51-4501-a713-1ecd1f3d4c65 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.121511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c3326-e38f-4c84-b16f-5fe2f115c747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.122401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c3326-e38f-4c84-b16f-5fe2f115c747 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.122792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c3326-e38f-4c84-b16f-5fe2f115c747 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.123177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c3326-e38f-4c84-b16f-5fe2f115c747 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.124127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c3326-e38f-4c84-b16f-5fe2f115c747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.126133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c3326-e38f-4c84-b16f-5fe2f115c747 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.139805 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 794/1587] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:16.141313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c3326-e38f-4c84-b16f-5fe2f115c747 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.147071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 794/1588] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:16.147766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5017aa5-7e51-4501-a713-1ecd1f3d4c65 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.160769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5017aa5-7e51-4501-a713-1ecd1f3d4c65 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.163659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c3326-e38f-4c84-b16f-5fe2f115c747 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.168867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63752-8410-409b-92df-ee67540bc72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.169994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63752-8410-409b-92df-ee67540bc72e admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.170477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63752-8410-409b-92df-ee67540bc72e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.170942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63752-8410-409b-92df-ee67540bc72e admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.172275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b63752-8410-409b-92df-ee67540bc72e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.172533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a960c6-31e8-4728-92d2-6906dcaa1edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.173401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a960c6-31e8-4728-92d2-6906dcaa1edd admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.173868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a960c6-31e8-4728-92d2-6906dcaa1edd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.174311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a960c6-31e8-4728-92d2-6906dcaa1edd admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.174760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b63752-8410-409b-92df-ee67540bc72e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.175308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a960c6-31e8-4728-92d2-6906dcaa1edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.178105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a960c6-31e8-4728-92d2-6906dcaa1edd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.192494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 795/1589] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:16.194069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63752-8410-409b-92df-ee67540bc72e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.218884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63752-8410-409b-92df-ee67540bc72e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.228422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 795/1590] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:16 2022] POST /identity/v3/users => generated 290 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:16.235256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a960c6-31e8-4728-92d2-6906dcaa1edd admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.239641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423be24-4a75-43e0-ab1f-7362d6009efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.241086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423be24-4a75-43e0-ab1f-7362d6009efe admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.241394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423be24-4a75-43e0-ab1f-7362d6009efe admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.241663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423be24-4a75-43e0-ab1f-7362d6009efe admin admin] PATH_INFO: `/v3/projects/ab2e51af12e6457dbacb69f4669fbfb6/users/327103627d35433b9c48a6dc8ca1978a/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.253267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a960c6-31e8-4728-92d2-6906dcaa1edd admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.262675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f9bef-2848-4a32-8774-72bfd8d35d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.263549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f9bef-2848-4a32-8774-72bfd8d35d9f admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.263895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f9bef-2848-4a32-8774-72bfd8d35d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.264233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f9bef-2848-4a32-8774-72bfd8d35d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.265116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f9bef-2848-4a32-8774-72bfd8d35d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.266920 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f9bef-2848-4a32-8774-72bfd8d35d9f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.268492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423be24-4a75-43e0-ab1f-7362d6009efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab2e51af12e6457dbacb69f4669fbfb6, user_id=327103627d35433b9c48a6dc8ca1978a, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.271326 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423be24-4a75-43e0-ab1f-7362d6009efe admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.281434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 796/1591] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:16.282776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f9bef-2848-4a32-8774-72bfd8d35d9f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.299600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 796/1592] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:16 2022] PUT /identity/v3/projects/ab2e51af12e6457dbacb69f4669fbfb6/users/327103627d35433b9c48a6dc8ca1978a/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:16.301856 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423be24-4a75-43e0-ab1f-7362d6009efe admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.304662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f9bef-2848-4a32-8774-72bfd8d35d9f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.402257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82002b81-1ba3-4f2b-81c8-e58a949428af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.402257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002b81-1ba3-4f2b-81c8-e58a949428af admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.402257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002b81-1ba3-4f2b-81c8-e58a949428af admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.402257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82002b81-1ba3-4f2b-81c8-e58a949428af admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.403062 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82002b81-1ba3-4f2b-81c8-e58a949428af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.406095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82002b81-1ba3-4f2b-81c8-e58a949428af admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.415304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423be24-4a75-43e0-ab1f-7362d6009efe admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.422069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb844d-dd98-4ffd-98b2-06cf0062da8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.422790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb844d-dd98-4ffd-98b2-06cf0062da8e admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.423078 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb844d-dd98-4ffd-98b2-06cf0062da8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.423357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb844d-dd98-4ffd-98b2-06cf0062da8e admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.424523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb844d-dd98-4ffd-98b2-06cf0062da8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.425167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 797/1593] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:16.426111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82002b81-1ba3-4f2b-81c8-e58a949428af admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.426780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb844d-dd98-4ffd-98b2-06cf0062da8e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.439027 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82002b81-1ba3-4f2b-81c8-e58a949428af admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.442321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 797/1594] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:16.443314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb844d-dd98-4ffd-98b2-06cf0062da8e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.451474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a7279-ba0d-4691-b0bd-875505336b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.452430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a7279-ba0d-4691-b0bd-875505336b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.452720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a7279-ba0d-4691-b0bd-875505336b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.453100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a7279-ba0d-4691-b0bd-875505336b8f admin admin] PATH_INFO: `/v3/projects/2143f0f2839945bc8ee444b86949393f/users/bf56654288144e2c873549a27f15e886/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.457076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:16.458400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb844d-dd98-4ffd-98b2-06cf0062da8e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.467641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d0285-05f1-45e8-997a-ddd9109dd1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.468733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d0285-05f1-45e8-997a-ddd9109dd1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.469227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d0285-05f1-45e8-997a-ddd9109dd1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.469731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d0285-05f1-45e8-997a-ddd9109dd1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.471498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d0285-05f1-45e8-997a-ddd9109dd1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.474969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d0285-05f1-45e8-997a-ddd9109dd1b9 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.480235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a7279-ba0d-4691-b0bd-875505336b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2143f0f2839945bc8ee444b86949393f, user_id=bf56654288144e2c873549a27f15e886, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.483197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a7279-ba0d-4691-b0bd-875505336b8f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.487323 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 798/1595] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:16.491973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d0285-05f1-45e8-997a-ddd9109dd1b9 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.510997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 798/1596] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:16 2022] PUT /identity/v3/projects/2143f0f2839945bc8ee444b86949393f/users/bf56654288144e2c873549a27f15e886/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:16.510997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a7279-ba0d-4691-b0bd-875505336b8f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.511455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d0285-05f1-45e8-997a-ddd9109dd1b9 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.528092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:16.606982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a7279-ba0d-4691-b0bd-875505336b8f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.611236 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972f7d2-c87b-42ba-aeb1-804268b11947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.612092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972f7d2-c87b-42ba-aeb1-804268b11947 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.612545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972f7d2-c87b-42ba-aeb1-804268b11947 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.612943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972f7d2-c87b-42ba-aeb1-804268b11947 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.613924 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972f7d2-c87b-42ba-aeb1-804268b11947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.615835 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972f7d2-c87b-42ba-aeb1-804268b11947 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.616507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be82f91-d3d7-486b-8277-86d579b4c9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.617559 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be82f91-d3d7-486b-8277-86d579b4c9bd admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.617832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be82f91-d3d7-486b-8277-86d579b4c9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.618114 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be82f91-d3d7-486b-8277-86d579b4c9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.618914 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be82f91-d3d7-486b-8277-86d579b4c9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.621407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be82f91-d3d7-486b-8277-86d579b4c9bd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.629678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 799/1597] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:16.629678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a8f83-a8b9-48d1-8363-a890ade3e2a3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.629678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a8f83-a8b9-48d1-8363-a890ade3e2a3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.629678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a8f83-a8b9-48d1-8363-a890ade3e2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.639389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 799/1598] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:16.640273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be82f91-d3d7-486b-8277-86d579b4c9bd admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.652902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be82f91-d3d7-486b-8277-86d579b4c9bd admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.659474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34919550-f7af-4c82-98a3-755c65482c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.660167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34919550-f7af-4c82-98a3-755c65482c5d admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.660558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34919550-f7af-4c82-98a3-755c65482c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.660836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34919550-f7af-4c82-98a3-755c65482c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.661578 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34919550-f7af-4c82-98a3-755c65482c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.663203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34919550-f7af-4c82-98a3-755c65482c5d admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.675882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 800/1599] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:16.677002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34919550-f7af-4c82-98a3-755c65482c5d admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.680629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3b6a8f83-a8b9-48d1-8363-a890ade3e2a3 None None] MFA Rules not processed for user `327103627d35433b9c48a6dc8ca1978a`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:16.682857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3b6a8f83-a8b9-48d1-8363-a890ade3e2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:16.692996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34919550-f7af-4c82-98a3-755c65482c5d admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.714681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262f9a3-3536-4e42-b259-89b2d8d64382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.716864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262f9a3-3536-4e42-b259-89b2d8d64382 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.717441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262f9a3-3536-4e42-b259-89b2d8d64382 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.718429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262f9a3-3536-4e42-b259-89b2d8d64382 admin admin] PATH_INFO: `/v3/projects/0c2f1232dd344ca7a48bf3fbc36dfb39/users/d4c6ceff33ee449193fe6d4f7b2c01ec/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.745757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262f9a3-3536-4e42-b259-89b2d8d64382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2f1232dd344ca7a48bf3fbc36dfb39, user_id=d4c6ceff33ee449193fe6d4f7b2c01ec, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.750279 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262f9a3-3536-4e42-b259-89b2d8d64382 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.775268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 801/1600] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:16 2022] PUT /identity/v3/projects/0c2f1232dd344ca7a48bf3fbc36dfb39/users/d4c6ceff33ee449193fe6d4f7b2c01ec/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:16.777979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262f9a3-3536-4e42-b259-89b2d8d64382 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.822126 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 800/1601] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:16 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:16.823568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ca635-783d-4693-a268-4fc48a2f4a1a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.824075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ca635-783d-4693-a268-4fc48a2f4a1a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.824639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ca635-783d-4693-a268-4fc48a2f4a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.877604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-234ca635-783d-4693-a268-4fc48a2f4a1a None None] MFA Rules not processed for user `bf56654288144e2c873549a27f15e886`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:16.879034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-234ca635-783d-4693-a268-4fc48a2f4a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:16.896025 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262f9a3-3536-4e42-b259-89b2d8d64382 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:16.904843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfc0bb-c256-45c1-bc4d-b335395d9e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:16.906279 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfc0bb-c256-45c1-bc4d-b335395d9e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:16.906775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfc0bb-c256-45c1-bc4d-b335395d9e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:16.907256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfc0bb-c256-45c1-bc4d-b335395d9e22 admin admin] PATH_INFO: `/v3/projects/dbff21a960e44c75be1cb0f63a5c6a15/users/c15bdb27a3be441d8e2a9812d50307e4/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:16.939351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfc0bb-c256-45c1-bc4d-b335395d9e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbff21a960e44c75be1cb0f63a5c6a15, user_id=c15bdb27a3be441d8e2a9812d50307e4, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:16.942315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfc0bb-c256-45c1-bc4d-b335395d9e22 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:16.959197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 802/1602] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:16 2022] PUT /identity/v3/projects/dbff21a960e44c75be1cb0f63a5c6a15/users/c15bdb27a3be441d8e2a9812d50307e4/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:16.960098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfc0bb-c256-45c1-bc4d-b335395d9e22 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:16.997016 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 801/1603] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:16 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:16.998069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ca635-783d-4693-a268-4fc48a2f4a1a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:17.010007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ca635-783d-4693-a268-4fc48a2f4a1a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:17.021440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:17.060007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:17.075687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfc0bb-c256-45c1-bc4d-b335395d9e22 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:17.082292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e51555-3390-4af2-a7d9-7f2f18164999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:17.083541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e51555-3390-4af2-a7d9-7f2f18164999 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.083948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e51555-3390-4af2-a7d9-7f2f18164999 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.084403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e51555-3390-4af2-a7d9-7f2f18164999 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.085227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e51555-3390-4af2-a7d9-7f2f18164999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:17.086869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e51555-3390-4af2-a7d9-7f2f18164999 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:17.102081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 803/1604] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:17.103038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e51555-3390-4af2-a7d9-7f2f18164999 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:17.114311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e51555-3390-4af2-a7d9-7f2f18164999 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:17.123182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2473b-4ab5-4c56-9dc8-9a98f1726ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:17.123722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8992f-b9a3-4065-835d-b3bdd99671db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:17.124649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2473b-4ab5-4c56-9dc8-9a98f1726ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.125110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2473b-4ab5-4c56-9dc8-9a98f1726ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.125295 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8992f-b9a3-4065-835d-b3bdd99671db admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.125574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8992f-b9a3-4065-835d-b3bdd99671db admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.125680 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2473b-4ab5-4c56-9dc8-9a98f1726ab6 admin admin] PATH_INFO: `/v3/projects/c9432177a8ce4a4abec0ac6e372ce681/users/1f026e791bf24a9fab1aaedb794e6229/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.126032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8992f-b9a3-4065-835d-b3bdd99671db admin admin] PATH_INFO: `/v3/projects/bc43a6a797fa4550be7d6e12c7a52ad7/users/844287bfecbf4527a8332b6527235e3f/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.151356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2473b-4ab5-4c56-9dc8-9a98f1726ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9432177a8ce4a4abec0ac6e372ce681, user_id=1f026e791bf24a9fab1aaedb794e6229, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:17.151872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8992f-b9a3-4065-835d-b3bdd99671db admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc43a6a797fa4550be7d6e12c7a52ad7, user_id=844287bfecbf4527a8332b6527235e3f, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:17.154127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2473b-4ab5-4c56-9dc8-9a98f1726ab6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:17.156169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8992f-b9a3-4065-835d-b3bdd99671db admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:17.188583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 802/1605] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:16 2022] PUT /identity/v3/projects/bc43a6a797fa4550be7d6e12c7a52ad7/users/844287bfecbf4527a8332b6527235e3f/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:17.188854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 804/1606] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:17 2022] PUT /identity/v3/projects/c9432177a8ce4a4abec0ac6e372ce681/users/1f026e791bf24a9fab1aaedb794e6229/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:17.188854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2473b-4ab5-4c56-9dc8-9a98f1726ab6 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:17.189682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848546d-1bda-4ff6-aa3b-ab3a21eab156 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.189682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848546d-1bda-4ff6-aa3b-ab3a21eab156 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.189682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848546d-1bda-4ff6-aa3b-ab3a21eab156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.225403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e848546d-1bda-4ff6-aa3b-ab3a21eab156 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:17.227559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e848546d-1bda-4ff6-aa3b-ab3a21eab156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:17.284427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2473b-4ab5-4c56-9dc8-9a98f1726ab6 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:17.337139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 803/1607] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:17.338856 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2ac5-e6a0-49db-be49-2e715d86660b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.339215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2ac5-e6a0-49db-be49-2e715d86660b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.339564 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2ac5-e6a0-49db-be49-2e715d86660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.378874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c533b0-5b9e-48e2-9f1a-bd4f1cac3ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:17.379859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c533b0-5b9e-48e2-9f1a-bd4f1cac3ada admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.380278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c533b0-5b9e-48e2-9f1a-bd4f1cac3ada admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.380766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c533b0-5b9e-48e2-9f1a-bd4f1cac3ada admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.381855 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c533b0-5b9e-48e2-9f1a-bd4f1cac3ada admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:17.385242 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c533b0-5b9e-48e2-9f1a-bd4f1cac3ada admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:17.386468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d49b2ac5-e6a0-49db-be49-2e715d86660b None None] MFA Rules not processed for user `c15bdb27a3be441d8e2a9812d50307e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:17.388132 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d49b2ac5-e6a0-49db-be49-2e715d86660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:17.424934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 805/1608] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:17 2022] POST /identity/v3/projects => generated 403 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:17.426103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c533b0-5b9e-48e2-9f1a-bd4f1cac3ada admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:17.481592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:17.506884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 804/1609] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:17.508200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b2ac5-e6a0-49db-be49-2e715d86660b None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:17.525849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c533b0-5b9e-48e2-9f1a-bd4f1cac3ada admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:17.526604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b2ac5-e6a0-49db-be49-2e715d86660b None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:17.535001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce83f2a-f6ed-4379-9963-6c2b6267f9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:17.535917 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce83f2a-f6ed-4379-9963-6c2b6267f9da admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.536351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce83f2a-f6ed-4379-9963-6c2b6267f9da admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.536779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce83f2a-f6ed-4379-9963-6c2b6267f9da admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.536943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446143ef-ca91-4ae5-a7c9-7287626b4fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:17.538294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce83f2a-f6ed-4379-9963-6c2b6267f9da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:17.538616 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446143ef-ca91-4ae5-a7c9-7287626b4fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.539261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446143ef-ca91-4ae5-a7c9-7287626b4fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.539939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446143ef-ca91-4ae5-a7c9-7287626b4fb2 admin admin] PATH_INFO: `/v3/projects/79723635097c40588d8dfedceb79ee68/users/284d7172360641a6b039d1bc23e1152e/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.541249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce83f2a-f6ed-4379-9963-6c2b6267f9da admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:17.568635 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446143ef-ca91-4ae5-a7c9-7287626b4fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79723635097c40588d8dfedceb79ee68, user_id=284d7172360641a6b039d1bc23e1152e, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:17.573779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446143ef-ca91-4ae5-a7c9-7287626b4fb2 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:17.589306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 806/1610] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:17 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:17.591590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b581ca-5586-4479-876f-e11b58d0837d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.592056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b581ca-5586-4479-876f-e11b58d0837d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.592547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b581ca-5586-4479-876f-e11b58d0837d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.594484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 805/1611] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:17 2022] PUT /identity/v3/projects/79723635097c40588d8dfedceb79ee68/users/284d7172360641a6b039d1bc23e1152e/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:17.604695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3276c-9c3e-478d-a57a-b1c2101d0c6a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.605129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3276c-9c3e-478d-a57a-b1c2101d0c6a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.605477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3276c-9c3e-478d-a57a-b1c2101d0c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.623422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-92b581ca-5586-4479-876f-e11b58d0837d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:17.625146 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-92b581ca-5586-4479-876f-e11b58d0837d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:17.645740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d9d3276c-9c3e-478d-a57a-b1c2101d0c6a None None] MFA Rules not processed for user `844287bfecbf4527a8332b6527235e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:17.648377 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d9d3276c-9c3e-478d-a57a-b1c2101d0c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:17.664983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:17.725440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 807/1612] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:17.727271 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b581ca-5586-4479-876f-e11b58d0837d None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:17.784207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 806/1613] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:17.785820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3276c-9c3e-478d-a57a-b1c2101d0c6a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:17.803158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3276c-9c3e-478d-a57a-b1c2101d0c6a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:17.810203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982de876-156c-4050-8900-80340778a4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:17.811464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982de876-156c-4050-8900-80340778a4fe admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.811926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982de876-156c-4050-8900-80340778a4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.812396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982de876-156c-4050-8900-80340778a4fe admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.813267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982de876-156c-4050-8900-80340778a4fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:17.816695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982de876-156c-4050-8900-80340778a4fe admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:17.817575 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b581ca-5586-4479-876f-e11b58d0837d None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:17.824917 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e79a0-5956-4316-ac97-dd363b301ddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:17.826043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e79a0-5956-4316-ac97-dd363b301ddf service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.826622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e79a0-5956-4316-ac97-dd363b301ddf service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.827161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e79a0-5956-4316-ac97-dd363b301ddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.847620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e79a0-5956-4316-ac97-dd363b301ddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:17.851039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e79a0-5956-4316-ac97-dd363b301ddf service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:17.861610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 807/1614] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:17 2022] POST /identity/v3/users => generated 308 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:17.863920 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d546fa-ec4c-4a5b-a992-45ea1c5cd1be None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.864307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d546fa-ec4c-4a5b-a992-45ea1c5cd1be None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.864719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d546fa-ec4c-4a5b-a992-45ea1c5cd1be None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.866718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 808/1615] 10.222.0.85 () {54 vars in 962 bytes} [Tue Jul 19 20:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:54:17.868074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d546fa-ec4c-4a5b-a992-45ea1c5cd1be None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:17.874061 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 808/1616] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:17.876237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886241e-273a-4f13-b1cb-87cb6fdd0810 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.877630 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886241e-273a-4f13-b1cb-87cb6fdd0810 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.878108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886241e-273a-4f13-b1cb-87cb6fdd0810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.886771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d546fa-ec4c-4a5b-a992-45ea1c5cd1be None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:17.894277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fb9ab-4c30-4188-b16e-c856bb74b6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:17.895114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fb9ab-4c30-4188-b16e-c856bb74b6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.895515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fb9ab-4c30-4188-b16e-c856bb74b6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.895902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080fb9ab-4c30-4188-b16e-c856bb74b6d7 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.897330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fb9ab-4c30-4188-b16e-c856bb74b6d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:17.899489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080fb9ab-4c30-4188-b16e-c856bb74b6d7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:17.912827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-9886241e-273a-4f13-b1cb-87cb6fdd0810 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:17.915530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9886241e-273a-4f13-b1cb-87cb6fdd0810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:17.943501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 809/1617] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:17 2022] POST /identity/v3/users => generated 290 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:17.943501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fb9ab-4c30-4188-b16e-c856bb74b6d7 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:17.946191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 809/1618] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:17.949257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4c648-3217-4e7e-a553-ec0d0a86db07 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.950005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4c648-3217-4e7e-a553-ec0d0a86db07 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.950725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4c648-3217-4e7e-a553-ec0d0a86db07 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.955092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 810/1619] 10.222.0.85 () {54 vars in 962 bytes} [Tue Jul 19 20:54:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:54:17.957270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4c648-3217-4e7e-a553-ec0d0a86db07 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:17.975344 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080fb9ab-4c30-4188-b16e-c856bb74b6d7 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:17.982303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c37aa5-3af1-456d-a6a7-230d8ff44edd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:17.983351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c37aa5-3af1-456d-a6a7-230d8ff44edd service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:17.984426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c37aa5-3af1-456d-a6a7-230d8ff44edd service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:17.985223 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c37aa5-3af1-456d-a6a7-230d8ff44edd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:17.986790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4c648-3217-4e7e-a553-ec0d0a86db07 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:18.001479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f65214-045e-489f-87c6-e9c7372a401c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:18.002601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f65214-045e-489f-87c6-e9c7372a401c admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:18.002964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c37aa5-3af1-456d-a6a7-230d8ff44edd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:18.003153 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f65214-045e-489f-87c6-e9c7372a401c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:18.003479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f65214-045e-489f-87c6-e9c7372a401c admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:18.005400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f65214-045e-489f-87c6-e9c7372a401c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:18.006469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c37aa5-3af1-456d-a6a7-230d8ff44edd service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:18.007966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f65214-045e-489f-87c6-e9c7372a401c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:18.027323 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 810/1620] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:18.031060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 811/1621] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:18.031060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f65214-045e-489f-87c6-e9c7372a401c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:18.031393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116709f5-5412-4ad6-a18b-30193087485d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:18.031393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116709f5-5412-4ad6-a18b-30193087485d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:18.031739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116709f5-5412-4ad6-a18b-30193087485d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:18.048175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f65214-045e-489f-87c6-e9c7372a401c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:18.056959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31abf3c-0c90-471f-b9cb-d2104868650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:18.058046 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31abf3c-0c90-471f-b9cb-d2104868650e admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:18.058506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31abf3c-0c90-471f-b9cb-d2104868650e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:18.058943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31abf3c-0c90-471f-b9cb-d2104868650e admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:18.060012 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31abf3c-0c90-471f-b9cb-d2104868650e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:18.063058 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31abf3c-0c90-471f-b9cb-d2104868650e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:18.085057 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 812/1622] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:18.086465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31abf3c-0c90-471f-b9cb-d2104868650e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:18.089875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f31abf3c-0c90-471f-b9cb-d2104868650e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:18.113144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-116709f5-5412-4ad6-a18b-30193087485d None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:18.114076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31abf3c-0c90-471f-b9cb-d2104868650e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:18.115217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-116709f5-5412-4ad6-a18b-30193087485d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:18.120222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038baba-34cf-45f3-be9b-78af7dab29e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:18.121281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038baba-34cf-45f3-be9b-78af7dab29e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:18.121767 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038baba-34cf-45f3-be9b-78af7dab29e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:18.122232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038baba-34cf-45f3-be9b-78af7dab29e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:18.139571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038baba-34cf-45f3-be9b-78af7dab29e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:18.142069 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038baba-34cf-45f3-be9b-78af7dab29e6 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:18.160572 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 813/1623] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:18.163093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f70af-eafe-4361-9dec-4060480bd45c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:18.163591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f70af-eafe-4361-9dec-4060480bd45c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:18.164081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f70af-eafe-4361-9dec-4060480bd45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:18.200833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-105f70af-eafe-4361-9dec-4060480bd45c None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:18.203256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-105f70af-eafe-4361-9dec-4060480bd45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:18.239720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:18.317471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 811/1624] 10.222.0.85 () {58 vars in 1059 bytes} [Tue Jul 19 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:18.318435 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116709f5-5412-4ad6-a18b-30193087485d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:18.338886 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 814/1625] 10.222.0.85 () {58 vars in 1059 bytes} [Tue Jul 19 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:18.340357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f70af-eafe-4361-9dec-4060480bd45c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:18.346069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116709f5-5412-4ad6-a18b-30193087485d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:18.365369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f70af-eafe-4361-9dec-4060480bd45c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:18.370473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9d972-708a-430f-bd78-c4e204e53549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:18.373629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9d972-708a-430f-bd78-c4e204e53549 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:18.374426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2ceee-6873-4bde-b1b7-74a6e93f373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:18.374645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9d972-708a-430f-bd78-c4e204e53549 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:18.375543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9d972-708a-430f-bd78-c4e204e53549 admin admin] PATH_INFO: `/v3/projects/e257784f35724cc28e1757a60f3bee64/users/4fcc8e82eb454f87a9292708af96fb3c/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:18.376175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2ceee-6873-4bde-b1b7-74a6e93f373c admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:18.376793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2ceee-6873-4bde-b1b7-74a6e93f373c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:18.377266 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2ceee-6873-4bde-b1b7-74a6e93f373c admin admin] PATH_INFO: `/v3/projects/f705095787924f88b9b118b431b1cf25/users/4122ae321c6041e395ab75cbe5621b07/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:18.407365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2ceee-6873-4bde-b1b7-74a6e93f373c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f705095787924f88b9b118b431b1cf25, user_id=4122ae321c6041e395ab75cbe5621b07, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:18.412896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2ceee-6873-4bde-b1b7-74a6e93f373c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:18.421247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9d972-708a-430f-bd78-c4e204e53549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e257784f35724cc28e1757a60f3bee64, user_id=4fcc8e82eb454f87a9292708af96fb3c, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:18.427158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9d972-708a-430f-bd78-c4e204e53549 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:18.441778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 815/1626] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:18 2022] PUT /identity/v3/projects/f705095787924f88b9b118b431b1cf25/users/4122ae321c6041e395ab75cbe5621b07/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:18.443737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2ceee-6873-4bde-b1b7-74a6e93f373c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:18.461348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 812/1627] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:18 2022] PUT /identity/v3/projects/e257784f35724cc28e1757a60f3bee64/users/4fcc8e82eb454f87a9292708af96fb3c/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:18.463640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9d972-708a-430f-bd78-c4e204e53549 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:18.503993 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:18.644569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2ceee-6873-4bde-b1b7-74a6e93f373c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:18.707585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9d972-708a-430f-bd78-c4e204e53549 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:18.726662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50e747-0f6e-4363-b66f-7033a5bfe837 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:18.729764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50e747-0f6e-4363-b66f-7033a5bfe837 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:18.733061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50e747-0f6e-4363-b66f-7033a5bfe837 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:18.733954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50e747-0f6e-4363-b66f-7033a5bfe837 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:18.846391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52d864-5d29-4d88-9b1c-7c33aa68f710 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:18.847733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52d864-5d29-4d88-9b1c-7c33aa68f710 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:18.848222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52d864-5d29-4d88-9b1c-7c33aa68f710 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:18.849309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52d864-5d29-4d88-9b1c-7c33aa68f710 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:18.899097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:18.908813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50e747-0f6e-4363-b66f-7033a5bfe837 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:18.911756 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50e747-0f6e-4363-b66f-7033a5bfe837 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:18.937584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 813/1628] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:18.941704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50e747-0f6e-4363-b66f-7033a5bfe837 service nova] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:19.043038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52d864-5d29-4d88-9b1c-7c33aa68f710 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:19.044356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50e747-0f6e-4363-b66f-7033a5bfe837 service nova] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:19.046305 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52d864-5d29-4d88-9b1c-7c33aa68f710 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:19.057156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813959b-bd4e-4f3b-8e0e-4fe80eeaded0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:19.057859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813959b-bd4e-4f3b-8e0e-4fe80eeaded0 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:19.058148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813959b-bd4e-4f3b-8e0e-4fe80eeaded0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:19.058432 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813959b-bd4e-4f3b-8e0e-4fe80eeaded0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:19.059519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813959b-bd4e-4f3b-8e0e-4fe80eeaded0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:19.061300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813959b-bd4e-4f3b-8e0e-4fe80eeaded0 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:19.066020 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 816/1629] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:19.067544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52d864-5d29-4d88-9b1c-7c33aa68f710 service nova] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:19.090256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 814/1630] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:19.091135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813959b-bd4e-4f3b-8e0e-4fe80eeaded0 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:19.093771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52d864-5d29-4d88-9b1c-7c33aa68f710 service nova] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:19.104344 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca02c74-9fd1-4f76-82bd-6b78007af258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:19.105561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca02c74-9fd1-4f76-82bd-6b78007af258 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:19.106258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca02c74-9fd1-4f76-82bd-6b78007af258 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:19.106941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca02c74-9fd1-4f76-82bd-6b78007af258 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:19.108780 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca02c74-9fd1-4f76-82bd-6b78007af258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:19.113091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca02c74-9fd1-4f76-82bd-6b78007af258 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:19.114420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813959b-bd4e-4f3b-8e0e-4fe80eeaded0 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:19.131685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83f819-4dad-4d09-977f-2240fe9cc145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:19.133297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83f819-4dad-4d09-977f-2240fe9cc145 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:19.134633 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83f819-4dad-4d09-977f-2240fe9cc145 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:19.135164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83f819-4dad-4d09-977f-2240fe9cc145 admin admin] PATH_INFO: `/v3/projects/f705095787924f88b9b118b431b1cf25/users/4122ae321c6041e395ab75cbe5621b07/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:19.138998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83f819-4dad-4d09-977f-2240fe9cc145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f705095787924f88b9b118b431b1cf25, user_id=4122ae321c6041e395ab75cbe5621b07, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:19.144694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83f819-4dad-4d09-977f-2240fe9cc145 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:19.158437 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 817/1631] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:19.164819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca02c74-9fd1-4f76-82bd-6b78007af258 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:19.175459 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 815/1632] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:19 2022] PUT /identity/v3/projects/f705095787924f88b9b118b431b1cf25/users/4122ae321c6041e395ab75cbe5621b07/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:19.186996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca02c74-9fd1-4f76-82bd-6b78007af258 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:19.197324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83528104-0a4e-4955-82c5-55ecf23e4ae3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:19.197901 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83528104-0a4e-4955-82c5-55ecf23e4ae3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:19.198675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83528104-0a4e-4955-82c5-55ecf23e4ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:19.283882 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-83528104-0a4e-4955-82c5-55ecf23e4ae3 None None] MFA Rules not processed for user `4122ae321c6041e395ab75cbe5621b07`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:19.290752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-83528104-0a4e-4955-82c5-55ecf23e4ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:19.373380 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a776868-fe3c-4161-98d8-ebab28b3040a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:19.374224 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a776868-fe3c-4161-98d8-ebab28b3040a admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:19.374508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a776868-fe3c-4161-98d8-ebab28b3040a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:19.374938 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a776868-fe3c-4161-98d8-ebab28b3040a admin admin] PATH_INFO: `/v3/projects/e257784f35724cc28e1757a60f3bee64/users/4fcc8e82eb454f87a9292708af96fb3c/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:19.379414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a776868-fe3c-4161-98d8-ebab28b3040a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e257784f35724cc28e1757a60f3bee64, user_id=4fcc8e82eb454f87a9292708af96fb3c, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:19.384044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a776868-fe3c-4161-98d8-ebab28b3040a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:19.410916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:19.412643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 818/1633] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:19 2022] PUT /identity/v3/projects/e257784f35724cc28e1757a60f3bee64/users/4fcc8e82eb454f87a9292708af96fb3c/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:19.432497 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975afc16-2b83-4579-aee8-802462f92a61 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:19.432814 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975afc16-2b83-4579-aee8-802462f92a61 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:19.433123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975afc16-2b83-4579-aee8-802462f92a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:19.569698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-975afc16-2b83-4579-aee8-802462f92a61 None None] MFA Rules not processed for user `4fcc8e82eb454f87a9292708af96fb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:19.571807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-975afc16-2b83-4579-aee8-802462f92a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:19.590731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 816/1634] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:19.615893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83528104-0a4e-4955-82c5-55ecf23e4ae3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:19.838031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 819/1635] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:19 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:19.855613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975afc16-2b83-4579-aee8-802462f92a61 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:19.872374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83528104-0a4e-4955-82c5-55ecf23e4ae3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:19.928030 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:20.011275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975afc16-2b83-4579-aee8-802462f92a61 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:20.031908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346677ea-52ee-4c75-bf4d-9ea45e67f628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:20.032616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346677ea-52ee-4c75-bf4d-9ea45e67f628 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:20.033153 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346677ea-52ee-4c75-bf4d-9ea45e67f628 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:20.033436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346677ea-52ee-4c75-bf4d-9ea45e67f628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:20.061662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346677ea-52ee-4c75-bf4d-9ea45e67f628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:20.073253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346677ea-52ee-4c75-bf4d-9ea45e67f628 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:20.079999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfaac9-cabf-473c-87e1-ed4befde9226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:20.080714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfaac9-cabf-473c-87e1-ed4befde9226 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:20.080999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfaac9-cabf-473c-87e1-ed4befde9226 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:20.081277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfaac9-cabf-473c-87e1-ed4befde9226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:20.116770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 820/1636] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:19 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:54:20.127269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9434b-57c0-46c9-b29c-31723660a53e None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:20.127631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9434b-57c0-46c9-b29c-31723660a53e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:20.130921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9434b-57c0-46c9-b29c-31723660a53e None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:20.130921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 821/1637] 10.222.0.85 () {54 vars in 912 bytes} [Tue Jul 19 20:54:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:54:20.142899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2cdf0-c00e-4068-ac29-8eaa8b76eeb9 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:20.143222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2cdf0-c00e-4068-ac29-8eaa8b76eeb9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:20.143589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2cdf0-c00e-4068-ac29-8eaa8b76eeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:20.248077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0bf2cdf0-c00e-4068-ac29-8eaa8b76eeb9 None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:20.251331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0bf2cdf0-c00e-4068-ac29-8eaa8b76eeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:20.386756 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfaac9-cabf-473c-87e1-ed4befde9226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:20.395170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfaac9-cabf-473c-87e1-ed4befde9226 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:20.447847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 817/1638] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 833 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:20.454049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff339a5-7fd8-4fed-aeb0-9f6d45f63b81 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:20.454557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff339a5-7fd8-4fed-aeb0-9f6d45f63b81 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:20.455286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff339a5-7fd8-4fed-aeb0-9f6d45f63b81 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:20.458871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 818/1639] 10.222.0.85 () {54 vars in 912 bytes} [Tue Jul 19 20:54:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:54:20.480940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1b9f2-0619-49e8-b1fc-21455a1c6cb1 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:20.481466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1b9f2-0619-49e8-b1fc-21455a1c6cb1 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:20.481972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1b9f2-0619-49e8-b1fc-21455a1c6cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:20.519600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-00b1b9f2-0619-49e8-b1fc-21455a1c6cb1 None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:20.521785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-00b1b9f2-0619-49e8-b1fc-21455a1c6cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:20.543760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 819/1640] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:20.564155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 822/1641] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:20.612549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2cdf0-c00e-4068-ac29-8eaa8b76eeb9 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:20.612788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1b9f2-0619-49e8-b1fc-21455a1c6cb1 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:20.647566 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2cdf0-c00e-4068-ac29-8eaa8b76eeb9 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:20.662530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:20.667424 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce8bc1-c4b1-4522-b9ba-dc6681f86b03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:20.668579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce8bc1-c4b1-4522-b9ba-dc6681f86b03 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:20.669227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce8bc1-c4b1-4522-b9ba-dc6681f86b03 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:20.669722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce8bc1-c4b1-4522-b9ba-dc6681f86b03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:20.709539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce8bc1-c4b1-4522-b9ba-dc6681f86b03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:20.716041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce8bc1-c4b1-4522-b9ba-dc6681f86b03 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:20.748631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 823/1642] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:20.763460 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce8bc1-c4b1-4522-b9ba-dc6681f86b03 service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:20.770782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1b9f2-0619-49e8-b1fc-21455a1c6cb1 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:20.780131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:20.781712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:20.782188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:20.782651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] PATH_INFO: `/v3/users/327103627d35433b9c48a6dc8ca1978a` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:20.785747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327103627d35433b9c48a6dc8ca1978a)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:20.789812 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:20.803043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce8bc1-c4b1-4522-b9ba-dc6681f86b03 service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:20.820944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d7344-79ca-4f88-a8a3-1ef6a1fe657b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:20.824321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d7344-79ca-4f88-a8a3-1ef6a1fe657b service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:20.825583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d7344-79ca-4f88-a8a3-1ef6a1fe657b service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:20.826267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d7344-79ca-4f88-a8a3-1ef6a1fe657b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:20.877627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d7344-79ca-4f88-a8a3-1ef6a1fe657b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:20.885567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d7344-79ca-4f88-a8a3-1ef6a1fe657b service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:20.932592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327103627d35433b9c48a6dc8ca1978a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:20.946338 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 824/1643] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:20.953748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327103627d35433b9c48a6dc8ca1978a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:20.976381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327103627d35433b9c48a6dc8ca1978a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:20.980696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327103627d35433b9c48a6dc8ca1978a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:21.025046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327103627d35433b9c48a6dc8ca1978a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:21.041028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 820/1644] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:20 2022] DELETE /identity/v3/users/327103627d35433b9c48a6dc8ca1978a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:21.046900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee30b-251e-4f1c-a4bf-2c3f2a8460dd None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:21.047461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee30b-251e-4f1c-a4bf-2c3f2a8460dd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:21.047841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee30b-251e-4f1c-a4bf-2c3f2a8460dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:21.104092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-daeee30b-251e-4f1c-a4bf-2c3f2a8460dd None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:21.106912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-daeee30b-251e-4f1c-a4bf-2c3f2a8460dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:21.326039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:21.329400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:21.353454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 825/1645] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:54:21.512672 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeee30b-251e-4f1c-a4bf-2c3f2a8460dd None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:21.516564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-daeee30b-251e-4f1c-a4bf-2c3f2a8460dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:21.558638 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e6c06-869c-4bbb-9314-bfad167d2879 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:21.573087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea139a16-6a34-42e4-992b-09b4b162e9a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:21.574751 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139a16-6a34-42e4-992b-09b4b162e9a8 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:21.575546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139a16-6a34-42e4-992b-09b4b162e9a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:21.576058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139a16-6a34-42e4-992b-09b4b162e9a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:21.579444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ea139a16-6a34-42e4-992b-09b4b162e9a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:21.679473 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:21.737788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeee30b-251e-4f1c-a4bf-2c3f2a8460dd None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:21.748491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53702451-f380-461c-b6bb-b4266397bada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:21.749098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702451-f380-461c-b6bb-b4266397bada service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:21.749521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702451-f380-461c-b6bb-b4266397bada service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:21.749807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702451-f380-461c-b6bb-b4266397bada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:21.778310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53702451-f380-461c-b6bb-b4266397bada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:21.780513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53702451-f380-461c-b6bb-b4266397bada service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:21.814475 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 826/1646] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:54:21.843123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:21.857970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea139a16-6a34-42e4-992b-09b4b162e9a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:21.865377 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea139a16-6a34-42e4-992b-09b4b162e9a8 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:21.891599 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 821/1647] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:54:24.319055 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53702451-f380-461c-b6bb-b4266397bada service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:24.322829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-53702451-f380-461c-b6bb-b4266397bada service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:24.402855 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53702451-f380-461c-b6bb-b4266397bada service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:24.448939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:24.450709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:24.451211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:24.451706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] PATH_INFO: `/v3/projects/ab2e51af12e6457dbacb69f4669fbfb6` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:24.481623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab2e51af12e6457dbacb69f4669fbfb6)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:24.498145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:24.725934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab2e51af12e6457dbacb69f4669fbfb6', 'request_id': 'req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:24.826112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab2e51af12e6457dbacb69f4669fbfb6', 'request_id': 'req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:24.894846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab2e51af12e6457dbacb69f4669fbfb6', 'request_id': 'req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:24.990761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] The token cache is being invalidate because project ab2e51af12e6457dbacb69f4669fbfb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:24.994205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:25.010897 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:25.020548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 827/1648] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:24 2022] DELETE /identity/v3/projects/ab2e51af12e6457dbacb69f4669fbfb6 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:25.023671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea139a16-6a34-42e4-992b-09b4b162e9a8 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:25.060982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea139a16-6a34-42e4-992b-09b4b162e9a8 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:25.325249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d00734-3344-4c48-b96b-819cd7bd8be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:25.332046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d00734-3344-4c48-b96b-819cd7bd8be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:25.344115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d00734-3344-4c48-b96b-819cd7bd8be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:25.344664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d00734-3344-4c48-b96b-819cd7bd8be8 admin admin] PATH_INFO: `/v3/users/4fcc8e82eb454f87a9292708af96fb3c` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:25.357383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d00734-3344-4c48-b96b-819cd7bd8be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fcc8e82eb454f87a9292708af96fb3c)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:25.367595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d00734-3344-4c48-b96b-819cd7bd8be8 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:25.633858 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f2d00734-3344-4c48-b96b-819cd7bd8be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fcc8e82eb454f87a9292708af96fb3c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:25.695747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f2d00734-3344-4c48-b96b-819cd7bd8be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fcc8e82eb454f87a9292708af96fb3c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:25.738080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f2d00734-3344-4c48-b96b-819cd7bd8be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fcc8e82eb454f87a9292708af96fb3c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:25.739580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f2d00734-3344-4c48-b96b-819cd7bd8be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fcc8e82eb454f87a9292708af96fb3c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:25.768309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f2d00734-3344-4c48-b96b-819cd7bd8be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fcc8e82eb454f87a9292708af96fb3c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:25.790910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 822/1649] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:25 2022] DELETE /identity/v3/users/4fcc8e82eb454f87a9292708af96fb3c => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:27.987504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:27.992478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:28.047770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8fa2e-75ef-4c10-bf5b-221a74cca5e2 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:28.259005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:28.260171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:28.260534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:28.261800 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] PATH_INFO: `/v3/users/bf56654288144e2c873549a27f15e886` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:28.266491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf56654288144e2c873549a27f15e886)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:28.269918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: WARNING py.warnings [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:54:28.269918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: warnings.warn(msg) Jul 19 20:54:28.269918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: Jul 19 20:54:28.271456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:28.519635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf56654288144e2c873549a27f15e886'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:28.545161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf56654288144e2c873549a27f15e886'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:28.546393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf56654288144e2c873549a27f15e886'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:28.586034 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:28.604290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf56654288144e2c873549a27f15e886'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:28.638727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf56654288144e2c873549a27f15e886'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:28.656783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 828/1650] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:27 2022] DELETE /identity/v3/users/bf56654288144e2c873549a27f15e886 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:28.667311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e29c10-97b9-4c03-9ffc-113b8972ec21 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:28.672654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e29c10-97b9-4c03-9ffc-113b8972ec21 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:28.674009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e29c10-97b9-4c03-9ffc-113b8972ec21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:28.759895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-a2e29c10-97b9-4c03-9ffc-113b8972ec21 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:28.763391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a2e29c10-97b9-4c03-9ffc-113b8972ec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:28.943347 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 823/1651] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:28.973032 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:28.975182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:29.138925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2ccd6-04a9-4b41-8478-9cc3df816c00 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:29.152261 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04060fe0-8f3f-4738-9c3b-4a83ac043188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:29.153120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04060fe0-8f3f-4738-9c3b-4a83ac043188 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:29.153418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04060fe0-8f3f-4738-9c3b-4a83ac043188 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:29.153761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04060fe0-8f3f-4738-9c3b-4a83ac043188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:29.178828 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04060fe0-8f3f-4738-9c3b-4a83ac043188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:29.180845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04060fe0-8f3f-4738-9c3b-4a83ac043188 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:29.215996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 829/1652] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:29.372071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e29c10-97b9-4c03-9ffc-113b8972ec21 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:29.374372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a2e29c10-97b9-4c03-9ffc-113b8972ec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:29.404063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e29c10-97b9-4c03-9ffc-113b8972ec21 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:29.432388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:29.433752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:29.436207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:29.436830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] PATH_INFO: `/v3/projects/e257784f35724cc28e1757a60f3bee64` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:29.439444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e257784f35724cc28e1757a60f3bee64)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:29.440770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: WARNING py.warnings [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 20:54:29.440770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: warnings.warn(msg) Jul 19 20:54:29.440770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: Jul 19 20:54:29.442329 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:29.458982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:29.618433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e257784f35724cc28e1757a60f3bee64', 'request_id': 'req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:29.637258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e257784f35724cc28e1757a60f3bee64', 'request_id': 'req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:29.659851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e257784f35724cc28e1757a60f3bee64', 'request_id': 'req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:29.679615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] The token cache is being invalidate because project e257784f35724cc28e1757a60f3bee64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:29.680655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:29.681677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c2151608-cf6a-4ce0-8dad-d00b1a29ec3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:29.683578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 824/1653] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:29 2022] DELETE /identity/v3/projects/e257784f35724cc28e1757a60f3bee64 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:29.692813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6535fa-7108-4308-add6-03d42db8abf4 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:29.693354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6535fa-7108-4308-add6-03d42db8abf4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:29.693860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6535fa-7108-4308-add6-03d42db8abf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:29.765694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ed6535fa-7108-4308-add6-03d42db8abf4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:29.769878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ed6535fa-7108-4308-add6-03d42db8abf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:30.228963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 830/1654] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 533 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:30.232011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f2ef5-579b-4c02-b264-a923ffbad000 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:30.233445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f2ef5-579b-4c02-b264-a923ffbad000 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:30.234122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f2ef5-579b-4c02-b264-a923ffbad000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:30.302336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c69f2ef5-579b-4c02-b264-a923ffbad000 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:30.310030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c69f2ef5-579b-4c02-b264-a923ffbad000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:30.354040 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 825/1655] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:30.356051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6535fa-7108-4308-add6-03d42db8abf4 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:30.401327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6535fa-7108-4308-add6-03d42db8abf4 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:30.422999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622a1d0-077f-480b-9087-8171ef0658cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:30.428817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622a1d0-077f-480b-9087-8171ef0658cc admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:30.430205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622a1d0-077f-480b-9087-8171ef0658cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:30.430873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622a1d0-077f-480b-9087-8171ef0658cc admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:30.434582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622a1d0-077f-480b-9087-8171ef0658cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:30.437636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622a1d0-077f-480b-9087-8171ef0658cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:30.447851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622a1d0-077f-480b-9087-8171ef0658cc admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:30.490016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 831/1656] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:30.495435 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f2ef5-579b-4c02-b264-a923ffbad000 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:30.524764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f2ef5-579b-4c02-b264-a923ffbad000 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:30.541638 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c108fd-89f3-4191-8fe0-5b2a8e799c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:30.542482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c108fd-89f3-4191-8fe0-5b2a8e799c74 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:30.542771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c108fd-89f3-4191-8fe0-5b2a8e799c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:30.543039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c108fd-89f3-4191-8fe0-5b2a8e799c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:30.543746 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c108fd-89f3-4191-8fe0-5b2a8e799c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:30.545920 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c108fd-89f3-4191-8fe0-5b2a8e799c74 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:30.607814 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 826/1657] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:30 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:54:30.612821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622a1d0-077f-480b-9087-8171ef0658cc admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:30.653832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622a1d0-077f-480b-9087-8171ef0658cc admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:30.838614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96014dd-bac3-460c-8336-42687148739d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:30.839515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96014dd-bac3-460c-8336-42687148739d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:30.839515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96014dd-bac3-460c-8336-42687148739d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:30.877906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285fca7-4059-441d-8fa7-9e890d08effe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:30.879668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285fca7-4059-441d-8fa7-9e890d08effe admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:30.880522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285fca7-4059-441d-8fa7-9e890d08effe admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:30.881567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285fca7-4059-441d-8fa7-9e890d08effe admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:30.883615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285fca7-4059-441d-8fa7-9e890d08effe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:30.890352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285fca7-4059-441d-8fa7-9e890d08effe admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:30.893269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d96014dd-bac3-460c-8336-42687148739d None None] MFA Rules not processed for user `d4c6ceff33ee449193fe6d4f7b2c01ec`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:30.895088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d96014dd-bac3-460c-8336-42687148739d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:31.029624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 832/1658] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:30 2022] POST /identity/v3/users => generated 302 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:31.036078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285fca7-4059-441d-8fa7-9e890d08effe admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:31.044634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:31.058912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285fca7-4059-441d-8fa7-9e890d08effe admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:31.066019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bddf6b-c7bf-4988-95b3-c1a6faaea4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:31.066735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bddf6b-c7bf-4988-95b3-c1a6faaea4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:31.067023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bddf6b-c7bf-4988-95b3-c1a6faaea4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:31.067306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bddf6b-c7bf-4988-95b3-c1a6faaea4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:31.068138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bddf6b-c7bf-4988-95b3-c1a6faaea4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:31.070307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bddf6b-c7bf-4988-95b3-c1a6faaea4a4 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:31.081140 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 827/1659] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:31.090944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 833/1660] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:31.096946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96014dd-bac3-460c-8336-42687148739d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:31.102885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d96014dd-bac3-460c-8336-42687148739d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:31.108479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bddf6b-c7bf-4988-95b3-c1a6faaea4a4 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:31.154134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bddf6b-c7bf-4988-95b3-c1a6faaea4a4 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:31.182037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950b140-f8ae-44cf-907d-7c75a667fde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:31.183061 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950b140-f8ae-44cf-907d-7c75a667fde7 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:31.183514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950b140-f8ae-44cf-907d-7c75a667fde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:31.183913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950b140-f8ae-44cf-907d-7c75a667fde7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:31.185238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950b140-f8ae-44cf-907d-7c75a667fde7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:31.191435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950b140-f8ae-44cf-907d-7c75a667fde7 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:31.247954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 834/1661] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:31.250929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950b140-f8ae-44cf-907d-7c75a667fde7 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:31.255413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8950b140-f8ae-44cf-907d-7c75a667fde7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:31.287095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950b140-f8ae-44cf-907d-7c75a667fde7 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:31.300169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:31.301982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:31.302890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:31.303315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] PATH_INFO: `/v3/projects/2143f0f2839945bc8ee444b86949393f` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:31.305147 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2143f0f2839945bc8ee444b86949393f)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:31.309645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:31.310318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:31.356081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96014dd-bac3-460c-8336-42687148739d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:31.372270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bb807-002d-4247-8f32-a1e303b43c18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:31.373128 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bb807-002d-4247-8f32-a1e303b43c18 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:31.373821 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bb807-002d-4247-8f32-a1e303b43c18 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:31.375567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bb807-002d-4247-8f32-a1e303b43c18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:31.399851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0bb807-002d-4247-8f32-a1e303b43c18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:31.405480 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0bb807-002d-4247-8f32-a1e303b43c18 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:31.432339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 828/1662] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:31.434428 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bb807-002d-4247-8f32-a1e303b43c18 service nova] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:31.457536 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2143f0f2839945bc8ee444b86949393f', 'request_id': 'req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:31.483871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bb807-002d-4247-8f32-a1e303b43c18 service nova] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:31.492275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2143f0f2839945bc8ee444b86949393f', 'request_id': 'req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:31.520926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2143f0f2839945bc8ee444b86949393f', 'request_id': 'req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:31.553406 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] The token cache is being invalidate because project 2143f0f2839945bc8ee444b86949393f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:31.553666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:31.554959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:31.560523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 835/1663] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:31 2022] DELETE /identity/v3/projects/2143f0f2839945bc8ee444b86949393f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:31.567208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:31.568883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:31.659914 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8187be-85a2-4e90-97a7-6cd9edf49906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:31.660966 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8187be-85a2-4e90-97a7-6cd9edf49906 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:31.662175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8187be-85a2-4e90-97a7-6cd9edf49906 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:31.663447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8187be-85a2-4e90-97a7-6cd9edf49906 admin admin] PATH_INFO: `/v3/projects/522264527e0e4130b615936b03e90e68/users/8a14450faa014e1ebae3fe7f8e07d936/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:31.701219 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8187be-85a2-4e90-97a7-6cd9edf49906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=522264527e0e4130b615936b03e90e68, user_id=8a14450faa014e1ebae3fe7f8e07d936, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:31.706610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8187be-85a2-4e90-97a7-6cd9edf49906 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:31.750963 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 829/1664] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:31 2022] PUT /identity/v3/projects/522264527e0e4130b615936b03e90e68/users/8a14450faa014e1ebae3fe7f8e07d936/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:31.756049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53843b38-2843-4a21-8ae0-a8d2389b6575 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:31.756661 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53843b38-2843-4a21-8ae0-a8d2389b6575 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:31.759388 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe873d-67e7-45a0-96b6-3cfb2eecbc0f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:31.760229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53843b38-2843-4a21-8ae0-a8d2389b6575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:31.812007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-53843b38-2843-4a21-8ae0-a8d2389b6575 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:31.815200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-53843b38-2843-4a21-8ae0-a8d2389b6575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:31.962763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:31.966127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:31.967260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:31.967739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] PATH_INFO: `/v3/users/4122ae321c6041e395ab75cbe5621b07` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:31.971710 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4122ae321c6041e395ab75cbe5621b07)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:31.976144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:32.013753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 830/1665] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:32.014665 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:32.094984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4122ae321c6041e395ab75cbe5621b07'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:32.114936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4122ae321c6041e395ab75cbe5621b07'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:32.115769 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4122ae321c6041e395ab75cbe5621b07'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:32.141761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53843b38-2843-4a21-8ae0-a8d2389b6575 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:32.144402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-53843b38-2843-4a21-8ae0-a8d2389b6575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:32.151130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4122ae321c6041e395ab75cbe5621b07'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:32.172495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4122ae321c6041e395ab75cbe5621b07'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:32.201180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 836/1666] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:31 2022] DELETE /identity/v3/users/4122ae321c6041e395ab75cbe5621b07 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:32.325996 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53843b38-2843-4a21-8ae0-a8d2389b6575 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:32.376072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:32.509492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6f614-c640-44c0-beeb-e5b74219e165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:32.510180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6f614-c640-44c0-beeb-e5b74219e165 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:32.510944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6f614-c640-44c0-beeb-e5b74219e165 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:32.511249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6f614-c640-44c0-beeb-e5b74219e165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:32.521709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-83a6f614-c640-44c0-beeb-e5b74219e165 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:32.672665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6f614-c640-44c0-beeb-e5b74219e165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:32.675471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6f614-c640-44c0-beeb-e5b74219e165 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:32.705289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 831/1667] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:32.777736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:32.779331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:32.818603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdeaa2-bb6c-4496-9257-0b207eb3b39d admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:32.862919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:32.867355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:32.871290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:32.879705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] PATH_INFO: `/v3/users/844287bfecbf4527a8332b6527235e3f` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:32.887858 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=844287bfecbf4527a8332b6527235e3f)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:32.916031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:33.078389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '844287bfecbf4527a8332b6527235e3f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:33.099972 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '844287bfecbf4527a8332b6527235e3f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:33.101303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '844287bfecbf4527a8332b6527235e3f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:33.175851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '844287bfecbf4527a8332b6527235e3f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:33.191591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '844287bfecbf4527a8332b6527235e3f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:33.199597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 837/1668] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:32 2022] DELETE /identity/v3/users/844287bfecbf4527a8332b6527235e3f => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:33.205626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716fa4c-d00f-4bb7-b50a-492ae560dfd2 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:33.206516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716fa4c-d00f-4bb7-b50a-492ae560dfd2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:33.206944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716fa4c-d00f-4bb7-b50a-492ae560dfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:33.237373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-6716fa4c-d00f-4bb7-b50a-492ae560dfd2 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:33.239376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6716fa4c-d00f-4bb7-b50a-492ae560dfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:33.542637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 832/1669] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:33.581574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:33.584464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:33.802561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8573008-8da3-457d-93e6-c6b9ca0ac6e5 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:33.827217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30bc106-cafa-495f-9e0d-5ef49f1e036b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:33.828010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bc106-cafa-495f-9e0d-5ef49f1e036b service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:33.828288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bc106-cafa-495f-9e0d-5ef49f1e036b service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:33.828604 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30bc106-cafa-495f-9e0d-5ef49f1e036b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:33.867021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bc106-cafa-495f-9e0d-5ef49f1e036b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:33.871776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30bc106-cafa-495f-9e0d-5ef49f1e036b service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:33.897489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 838/1670] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:33.904944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716fa4c-d00f-4bb7-b50a-492ae560dfd2 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:33.907952 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6716fa4c-d00f-4bb7-b50a-492ae560dfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:33.933273 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716fa4c-d00f-4bb7-b50a-492ae560dfd2 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:33.946767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:33.949988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:33.951116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:33.952274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] PATH_INFO: `/v3/projects/f705095787924f88b9b118b431b1cf25` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:33.955366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f705095787924f88b9b118b431b1cf25)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:33.967783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:34.174044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f705095787924f88b9b118b431b1cf25', 'request_id': 'req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:34.197364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f705095787924f88b9b118b431b1cf25', 'request_id': 'req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:34.226349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f705095787924f88b9b118b431b1cf25', 'request_id': 'req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:34.244180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:34.264529 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] The token cache is being invalidate because project f705095787924f88b9b118b431b1cf25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:34.267900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:34.277142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:34.289513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 833/1671] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:33 2022] DELETE /identity/v3/projects/f705095787924f88b9b118b431b1cf25 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:34.292430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964fb5f-7723-4ea9-a8b0-0fc46f486232 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:34.292975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964fb5f-7723-4ea9-a8b0-0fc46f486232 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:34.293471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964fb5f-7723-4ea9-a8b0-0fc46f486232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:34.319370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:34.385799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e964fb5f-7723-4ea9-a8b0-0fc46f486232 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:34.393181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e964fb5f-7723-4ea9-a8b0-0fc46f486232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:34.733707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:34.737177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:34.777088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 839/1672] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:34.798918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3b94d-041c-42d1-b054-5bf94d6a1a3d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:34.799390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3b94d-041c-42d1-b054-5bf94d6a1a3d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:34.799875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3b94d-041c-42d1-b054-5bf94d6a1a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:34.880863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-38c3b94d-041c-42d1-b054-5bf94d6a1a3d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:34.883514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-38c3b94d-041c-42d1-b054-5bf94d6a1a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:34.898865 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 840/1673] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:34.904813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3b94d-041c-42d1-b054-5bf94d6a1a3d None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:34.935678 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3b94d-041c-42d1-b054-5bf94d6a1a3d None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:34.949036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8a099-c14d-48ba-a464-dcee6ca4e978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:34.950333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8a099-c14d-48ba-a464-dcee6ca4e978 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:34.951249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8a099-c14d-48ba-a464-dcee6ca4e978 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:34.951787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8a099-c14d-48ba-a464-dcee6ca4e978 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:34.953102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8a099-c14d-48ba-a464-dcee6ca4e978 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:34.953686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8a099-c14d-48ba-a464-dcee6ca4e978 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:34.957558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8a099-c14d-48ba-a464-dcee6ca4e978 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:34.983354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 841/1674] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:34.986247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8a099-c14d-48ba-a464-dcee6ca4e978 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:35.019832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8a099-c14d-48ba-a464-dcee6ca4e978 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:35.031788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ae871-f970-48a2-9590-fc7847c532c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:35.032592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ae871-f970-48a2-9590-fc7847c532c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:35.034226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ae871-f970-48a2-9590-fc7847c532c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:35.034518 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ae871-f970-48a2-9590-fc7847c532c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:35.036989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ae871-f970-48a2-9590-fc7847c532c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:35.039261 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ae871-f970-48a2-9590-fc7847c532c8 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:35.122481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4b5a2-bdb2-47fd-bb7e-6c75e8cd255d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:35.146203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 842/1675] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:34 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:35.149553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ae871-f970-48a2-9590-fc7847c532c8 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:35.150252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ed987-59b4-40d6-8eb4-597920c05e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:35.152095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ed987-59b4-40d6-8eb4-597920c05e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:35.153043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ed987-59b4-40d6-8eb4-597920c05e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:35.153743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ed987-59b4-40d6-8eb4-597920c05e1f admin admin] PATH_INFO: `/v3/users/c15bdb27a3be441d8e2a9812d50307e4` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:35.157656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ed987-59b4-40d6-8eb4-597920c05e1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c15bdb27a3be441d8e2a9812d50307e4)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:35.163734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ed987-59b4-40d6-8eb4-597920c05e1f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:35.324744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de7ed987-59b4-40d6-8eb4-597920c05e1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c15bdb27a3be441d8e2a9812d50307e4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:35.337378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de7ed987-59b4-40d6-8eb4-597920c05e1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c15bdb27a3be441d8e2a9812d50307e4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:35.368749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de7ed987-59b4-40d6-8eb4-597920c05e1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c15bdb27a3be441d8e2a9812d50307e4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:35.369666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de7ed987-59b4-40d6-8eb4-597920c05e1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c15bdb27a3be441d8e2a9812d50307e4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:35.407269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de7ed987-59b4-40d6-8eb4-597920c05e1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c15bdb27a3be441d8e2a9812d50307e4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:35.420288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 834/1676] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:34 2022] DELETE /identity/v3/users/c15bdb27a3be441d8e2a9812d50307e4 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:35.429967 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275ca4b-cd52-44c3-a9ac-1997b9a970ca None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:35.430653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275ca4b-cd52-44c3-a9ac-1997b9a970ca None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:35.431153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275ca4b-cd52-44c3-a9ac-1997b9a970ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:35.441119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ae871-f970-48a2-9590-fc7847c532c8 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:35.464156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c275ca4b-cd52-44c3-a9ac-1997b9a970ca None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:35.466028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c275ca4b-cd52-44c3-a9ac-1997b9a970ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:35.675157 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a96268-c1c2-4d8a-b096-5b7b7758e9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:35.680244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a96268-c1c2-4d8a-b096-5b7b7758e9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:35.681575 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a96268-c1c2-4d8a-b096-5b7b7758e9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:35.683294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a96268-c1c2-4d8a-b096-5b7b7758e9b2 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:35.684564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a96268-c1c2-4d8a-b096-5b7b7758e9b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:35.690793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a96268-c1c2-4d8a-b096-5b7b7758e9b2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:35.740115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 835/1677] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:35.770026 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:35.796941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275ca4b-cd52-44c3-a9ac-1997b9a970ca None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:35.799672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c275ca4b-cd52-44c3-a9ac-1997b9a970ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:35.801265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 843/1678] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:35 2022] POST /identity/v3/users => generated 291 bytes in 652 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:35.805160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a96268-c1c2-4d8a-b096-5b7b7758e9b2 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:35.807471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-22a96268-c1c2-4d8a-b096-5b7b7758e9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:35.830697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a96268-c1c2-4d8a-b096-5b7b7758e9b2 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:35.842154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad85c6-7ade-4855-b407-cde39654f6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:35.843793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad85c6-7ade-4855-b407-cde39654f6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:35.844386 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad85c6-7ade-4855-b407-cde39654f6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:35.844861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad85c6-7ade-4855-b407-cde39654f6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:35.847084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad85c6-7ade-4855-b407-cde39654f6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:35.849764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad85c6-7ade-4855-b407-cde39654f6b9 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:35.872202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 844/1679] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:35.875425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:35.879972 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad85c6-7ade-4855-b407-cde39654f6b9 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:35.910748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad85c6-7ade-4855-b407-cde39654f6b9 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:35.919999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73dbd-7560-4f56-ad8f-c6de38bf9792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:35.920991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73dbd-7560-4f56-ad8f-c6de38bf9792 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:35.921285 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73dbd-7560-4f56-ad8f-c6de38bf9792 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:35.921568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73dbd-7560-4f56-ad8f-c6de38bf9792 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:35.923583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d73dbd-7560-4f56-ad8f-c6de38bf9792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:35.929914 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d73dbd-7560-4f56-ad8f-c6de38bf9792 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:35.960305 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 845/1680] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:35.967705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73dbd-7560-4f56-ad8f-c6de38bf9792 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:36.024321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73dbd-7560-4f56-ad8f-c6de38bf9792 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:36.031328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275ca4b-cd52-44c3-a9ac-1997b9a970ca None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:36.041540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7b4ab-0241-409e-9b20-d3d343ab2ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:36.041780 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5b9a5-3e29-4500-8f22-583255b44c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:36.041881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7b4ab-0241-409e-9b20-d3d343ab2ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:36.041881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7b4ab-0241-409e-9b20-d3d343ab2ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:36.042003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5b9a5-3e29-4500-8f22-583255b44c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:36.042264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5b9a5-3e29-4500-8f22-583255b44c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:36.043079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5b9a5-3e29-4500-8f22-583255b44c04 admin admin] PATH_INFO: `/v3/projects/785f77170a324c33833523407884657f/users/bffae7676011443e8c95a0c6a66a2fc4/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:36.043590 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7b4ab-0241-409e-9b20-d3d343ab2ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:36.068465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7b4ab-0241-409e-9b20-d3d343ab2ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:36.072601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7b4ab-0241-409e-9b20-d3d343ab2ee0 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:36.089964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5b9a5-3e29-4500-8f22-583255b44c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=785f77170a324c33833523407884657f, user_id=bffae7676011443e8c95a0c6a66a2fc4, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:36.093464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5b9a5-3e29-4500-8f22-583255b44c04 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:36.102750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 836/1681] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:36.104665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7b4ab-0241-409e-9b20-d3d343ab2ee0 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:36.109838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2fc7b4ab-0241-409e-9b20-d3d343ab2ee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:36.119958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 846/1682] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:35 2022] PUT /identity/v3/projects/785f77170a324c33833523407884657f/users/bffae7676011443e8c95a0c6a66a2fc4/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:36.146067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176fe0b-8b6f-41bd-82e2-bb435cabada2 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:36.147193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176fe0b-8b6f-41bd-82e2-bb435cabada2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:36.148602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176fe0b-8b6f-41bd-82e2-bb435cabada2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:36.168408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7b4ab-0241-409e-9b20-d3d343ab2ee0 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:36.251361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-3176fe0b-8b6f-41bd-82e2-bb435cabada2 None None] MFA Rules not processed for user `bffae7676011443e8c95a0c6a66a2fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:36.253054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3176fe0b-8b6f-41bd-82e2-bb435cabada2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:36.408540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:36.409471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:36.409906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:36.410185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] PATH_INFO: `/v3/projects/bc43a6a797fa4550be7d6e12c7a52ad7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:36.413532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc43a6a797fa4550be7d6e12c7a52ad7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:36.427592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:36.436205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 847/1683] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:36.442040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176fe0b-8b6f-41bd-82e2-bb435cabada2 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:36.476451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176fe0b-8b6f-41bd-82e2-bb435cabada2 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:36.492847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0388156-3669-428f-b279-bb65d9b7e7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:36.493616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0388156-3669-428f-b279-bb65d9b7e7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:36.494080 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0388156-3669-428f-b279-bb65d9b7e7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:36.494253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0388156-3669-428f-b279-bb65d9b7e7f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:36.494961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0388156-3669-428f-b279-bb65d9b7e7f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:36.504282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0388156-3669-428f-b279-bb65d9b7e7f2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:36.566632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 848/1684] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:36 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:36.571278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0388156-3669-428f-b279-bb65d9b7e7f2 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:36.587265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc43a6a797fa4550be7d6e12c7a52ad7', 'request_id': 'req-34cb015c-2d0e-4d42-8767-76ec85403e65'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:36.604941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc43a6a797fa4550be7d6e12c7a52ad7', 'request_id': 'req-34cb015c-2d0e-4d42-8767-76ec85403e65'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:36.625683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc43a6a797fa4550be7d6e12c7a52ad7', 'request_id': 'req-34cb015c-2d0e-4d42-8767-76ec85403e65'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:36.659536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] The token cache is being invalidate because project bc43a6a797fa4550be7d6e12c7a52ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:36.661970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:36.664747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-34cb015c-2d0e-4d42-8767-76ec85403e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:36.669268 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 837/1685] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:36 2022] DELETE /identity/v3/projects/bc43a6a797fa4550be7d6e12c7a52ad7 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:36.678675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6540-d617-4dd9-9225-ab83324cb4d3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:36.681545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6540-d617-4dd9-9225-ab83324cb4d3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:36.689450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6540-d617-4dd9-9225-ab83324cb4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:36.795281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-ddaa6540-d617-4dd9-9225-ab83324cb4d3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:36.796546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ddaa6540-d617-4dd9-9225-ab83324cb4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:36.856582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0388156-3669-428f-b279-bb65d9b7e7f2 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:36.893413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cf259-3018-444c-a0e8-405b341216f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:36.894182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cf259-3018-444c-a0e8-405b341216f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:36.894475 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cf259-3018-444c-a0e8-405b341216f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:36.894760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cf259-3018-444c-a0e8-405b341216f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:36.897718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cf259-3018-444c-a0e8-405b341216f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:36.907886 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1cf259-3018-444c-a0e8-405b341216f0 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:37.011149 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 849/1686] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:36 2022] POST /identity/v3/users => generated 296 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:37.014295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cf259-3018-444c-a0e8-405b341216f0 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:37.023685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2c1cf259-3018-444c-a0e8-405b341216f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:37.040516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:37.067897 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cf259-3018-444c-a0e8-405b341216f0 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:37.088727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 838/1687] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:37.091646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11ef9f-d454-4c22-851d-f6124ab629d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:37.093912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11ef9f-d454-4c22-851d-f6124ab629d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:37.094542 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11ef9f-d454-4c22-851d-f6124ab629d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:37.095280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11ef9f-d454-4c22-851d-f6124ab629d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:37.096516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11ef9f-d454-4c22-851d-f6124ab629d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:37.099320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11ef9f-d454-4c22-851d-f6124ab629d6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:37.108185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:37.108921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08226fc4-81ae-4d02-8754-f9712a1a4837 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:37.111772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08226fc4-81ae-4d02-8754-f9712a1a4837 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:37.112195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08226fc4-81ae-4d02-8754-f9712a1a4837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:37.124462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 850/1688] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:37.125513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11ef9f-d454-4c22-851d-f6124ab629d6 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:37.155812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11ef9f-d454-4c22-851d-f6124ab629d6 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:37.160225 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-08226fc4-81ae-4d02-8754-f9712a1a4837 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:37.162233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-08226fc4-81ae-4d02-8754-f9712a1a4837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:37.170790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00ef2d-fbd0-4224-b5f2-0cf651176fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:37.171833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00ef2d-fbd0-4224-b5f2-0cf651176fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:37.172202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00ef2d-fbd0-4224-b5f2-0cf651176fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:37.173567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00ef2d-fbd0-4224-b5f2-0cf651176fbd admin admin] PATH_INFO: `/v3/projects/242ba4b38fd24d388fbddda34faf1881/users/60cdf320996d45a8bf9d9623044ad0aa/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:37.192523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 839/1689] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:37.197910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08226fc4-81ae-4d02-8754-f9712a1a4837 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:37.219123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08226fc4-81ae-4d02-8754-f9712a1a4837 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:37.221607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00ef2d-fbd0-4224-b5f2-0cf651176fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=242ba4b38fd24d388fbddda34faf1881, user_id=60cdf320996d45a8bf9d9623044ad0aa, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:37.228251 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00ef2d-fbd0-4224-b5f2-0cf651176fbd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:37.233782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4efd21-0fd5-4886-b6c9-220b451bbf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:37.234476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4efd21-0fd5-4886-b6c9-220b451bbf96 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:37.234763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4efd21-0fd5-4886-b6c9-220b451bbf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:37.237470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4efd21-0fd5-4886-b6c9-220b451bbf96 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:37.239754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4efd21-0fd5-4886-b6c9-220b451bbf96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:37.240068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4efd21-0fd5-4886-b6c9-220b451bbf96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:37.242185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4efd21-0fd5-4886-b6c9-220b451bbf96 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:37.268959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 851/1690] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:37 2022] PUT /identity/v3/projects/242ba4b38fd24d388fbddda34faf1881/users/60cdf320996d45a8bf9d9623044ad0aa/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:37.274291 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00ef2d-fbd0-4224-b5f2-0cf651176fbd admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:37.311413 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 840/1691] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:37.313037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e09e73-cf8a-4c78-89f0-53dca040b279 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:37.313333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e09e73-cf8a-4c78-89f0-53dca040b279 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:37.315056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e09e73-cf8a-4c78-89f0-53dca040b279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:37.398927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-35e09e73-cf8a-4c78-89f0-53dca040b279 None None] MFA Rules not processed for user `1f026e791bf24a9fab1aaedb794e6229`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:37.400203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-35e09e73-cf8a-4c78-89f0-53dca040b279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:37.491372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00ef2d-fbd0-4224-b5f2-0cf651176fbd admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:37.501511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebef686-eab2-4955-bba2-21681b5965cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:37.503827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebef686-eab2-4955-bba2-21681b5965cd admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:37.504558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebef686-eab2-4955-bba2-21681b5965cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:37.505174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebef686-eab2-4955-bba2-21681b5965cd admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:37.506823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebef686-eab2-4955-bba2-21681b5965cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:37.512820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebef686-eab2-4955-bba2-21681b5965cd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:37.529520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 841/1692] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:37.542988 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 852/1693] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:37.545188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e09e73-cf8a-4c78-89f0-53dca040b279 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:37.547986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebef686-eab2-4955-bba2-21681b5965cd admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:37.551798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4ebef686-eab2-4955-bba2-21681b5965cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:37.558786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e09e73-cf8a-4c78-89f0-53dca040b279 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:37.564142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d8a4c-7247-4f1e-bd0f-e44009ade415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:37.564983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d8a4c-7247-4f1e-bd0f-e44009ade415 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:37.565401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d8a4c-7247-4f1e-bd0f-e44009ade415 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:37.565784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d8a4c-7247-4f1e-bd0f-e44009ade415 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:37.566652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d8a4c-7247-4f1e-bd0f-e44009ade415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:37.569009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d8a4c-7247-4f1e-bd0f-e44009ade415 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:37.618473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 842/1694] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:37 2022] POST /identity/v3/projects => generated 393 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:37.619600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d8a4c-7247-4f1e-bd0f-e44009ade415 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:37.680564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebef686-eab2-4955-bba2-21681b5965cd admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:37.796508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d8a4c-7247-4f1e-bd0f-e44009ade415 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:37.809688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e0452-5854-4f09-9d22-e64be9acf91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:37.811912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e0452-5854-4f09-9d22-e64be9acf91e admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:37.812379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e0452-5854-4f09-9d22-e64be9acf91e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:37.812715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e0452-5854-4f09-9d22-e64be9acf91e admin admin] PATH_INFO: `/v3/projects/242ba4b38fd24d388fbddda34faf1881/users/60cdf320996d45a8bf9d9623044ad0aa/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:37.821612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e0452-5854-4f09-9d22-e64be9acf91e admin admin] RBAC: Authorizing `identity:create_grant(project_id=242ba4b38fd24d388fbddda34faf1881, user_id=60cdf320996d45a8bf9d9623044ad0aa, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:37.825447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e0452-5854-4f09-9d22-e64be9acf91e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:37.865672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 843/1695] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:37 2022] PUT /identity/v3/projects/242ba4b38fd24d388fbddda34faf1881/users/60cdf320996d45a8bf9d9623044ad0aa/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:37.867026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e0452-5854-4f09-9d22-e64be9acf91e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:37.946521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a194b-6199-46bb-b0fa-7c1761318652 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:37.948752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a194b-6199-46bb-b0fa-7c1761318652 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:37.950136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a194b-6199-46bb-b0fa-7c1761318652 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:37.950406 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a194b-6199-46bb-b0fa-7c1761318652 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:37.969554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:37.985324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:38.081094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a194b-6199-46bb-b0fa-7c1761318652 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:38.083611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a194b-6199-46bb-b0fa-7c1761318652 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:38.099233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 853/1696] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:38.102199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc275a-b0c9-426c-905b-b381176a689f None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:38.102863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc275a-b0c9-426c-905b-b381176a689f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:38.103356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc275a-b0c9-426c-905b-b381176a689f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:38.132156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e0452-5854-4f09-9d22-e64be9acf91e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:38.145676 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-cfdc275a-b0c9-426c-905b-b381176a689f None None] MFA Rules not processed for user `60cdf320996d45a8bf9d9623044ad0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:38.147317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-cfdc275a-b0c9-426c-905b-b381176a689f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:38.151446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1a873-d2d6-420f-9ec8-b740d311df10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:38.152566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1a873-d2d6-420f-9ec8-b740d311df10 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:38.153029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1a873-d2d6-420f-9ec8-b740d311df10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:38.153489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1a873-d2d6-420f-9ec8-b740d311df10 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:38.154375 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1a873-d2d6-420f-9ec8-b740d311df10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:38.156587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1a873-d2d6-420f-9ec8-b740d311df10 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:38.198571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 844/1697] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:37 2022] POST /identity/v3/users => generated 297 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:38.205206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1a873-d2d6-420f-9ec8-b740d311df10 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:38.216056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cda1a873-d2d6-420f-9ec8-b740d311df10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:38.249790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1a873-d2d6-420f-9ec8-b740d311df10 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:38.265688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:38.268767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:38.269179 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:38.269543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] PATH_INFO: `/v3/projects/dbff21a960e44c75be1cb0f63a5c6a15` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:38.271810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbff21a960e44c75be1cb0f63a5c6a15)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:38.273910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:38.347817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 854/1698] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:38.350075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc275a-b0c9-426c-905b-b381176a689f None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:38.385759 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbff21a960e44c75be1cb0f63a5c6a15', 'request_id': 'req-506fa8f5-4a62-4aee-8038-7d26744b3245'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:38.396155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc275a-b0c9-426c-905b-b381176a689f None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:38.402304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbff21a960e44c75be1cb0f63a5c6a15', 'request_id': 'req-506fa8f5-4a62-4aee-8038-7d26744b3245'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:38.416574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbff21a960e44c75be1cb0f63a5c6a15', 'request_id': 'req-506fa8f5-4a62-4aee-8038-7d26744b3245'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:38.453660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] The token cache is being invalidate because project dbff21a960e44c75be1cb0f63a5c6a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:38.454105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:38.455299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:38.462037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 845/1699] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:38 2022] DELETE /identity/v3/projects/dbff21a960e44c75be1cb0f63a5c6a15 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:38.462832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:38.468526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:38.594474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fbe7f-2f0d-4387-9c2e-5f677898bd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:38.597159 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fbe7f-2f0d-4387-9c2e-5f677898bd9a admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:38.599712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fbe7f-2f0d-4387-9c2e-5f677898bd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:38.600468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fbe7f-2f0d-4387-9c2e-5f677898bd9a admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:38.610988 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fbe7f-2f0d-4387-9c2e-5f677898bd9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:38.618697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71fbe7f-2f0d-4387-9c2e-5f677898bd9a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:38.655930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 855/1700] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:38.659521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760699d-9f20-4b56-9c5a-abfc28555e63 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:38.660264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760699d-9f20-4b56-9c5a-abfc28555e63 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:38.660690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760699d-9f20-4b56-9c5a-abfc28555e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:38.722032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fa8f5-4a62-4aee-8038-7d26744b3245 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:38.727251 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-7760699d-9f20-4b56-9c5a-abfc28555e63 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:38.729466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7760699d-9f20-4b56-9c5a-abfc28555e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:38.745254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89f60c-4073-4109-9cc6-4515be123b7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:38.746340 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f60c-4073-4109-9cc6-4515be123b7f service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:38.746752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f60c-4073-4109-9cc6-4515be123b7f service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:38.747246 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f60c-4073-4109-9cc6-4515be123b7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:38.754060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1c89f60c-4073-4109-9cc6-4515be123b7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:38.755578 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 856/1701] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:38.766547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760699d-9f20-4b56-9c5a-abfc28555e63 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:38.775852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7760699d-9f20-4b56-9c5a-abfc28555e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:38.803538 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760699d-9f20-4b56-9c5a-abfc28555e63 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:38.821465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9244480-df08-439a-837e-1b325f7a9fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:38.822337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9244480-df08-439a-837e-1b325f7a9fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:38.822654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9244480-df08-439a-837e-1b325f7a9fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:38.822995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9244480-df08-439a-837e-1b325f7a9fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:38.823955 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9244480-df08-439a-837e-1b325f7a9fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:38.830218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9244480-df08-439a-837e-1b325f7a9fa3 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:38.877221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 857/1702] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:38.878960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9244480-df08-439a-837e-1b325f7a9fa3 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:38.890435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e9244480-df08-439a-837e-1b325f7a9fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:38.931688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9244480-df08-439a-837e-1b325f7a9fa3 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:38.944152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f475d-2025-439f-9922-217b4b38c830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:38.945611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f475d-2025-439f-9922-217b4b38c830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:38.945983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f475d-2025-439f-9922-217b4b38c830 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:38.946531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f475d-2025-439f-9922-217b4b38c830 admin admin] PATH_INFO: `/v3/users/000d34f0bcd1443fa703cd68bded1c5e` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:38.948355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f475d-2025-439f-9922-217b4b38c830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=000d34f0bcd1443fa703cd68bded1c5e)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:38.951505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f475d-2025-439f-9922-217b4b38c830 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:38.961267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:38.989358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:39.006952 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89f60c-4073-4109-9cc6-4515be123b7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:39.008679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89f60c-4073-4109-9cc6-4515be123b7f service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:39.051657 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 846/1703] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:39.068481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36b4e5-6dce-41f8-a3d0-45433be1eab7 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:39.068824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36b4e5-6dce-41f8-a3d0-45433be1eab7 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:39.069046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36b4e5-6dce-41f8-a3d0-45433be1eab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:39.111664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0f4f475d-2025-439f-9922-217b4b38c830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '000d34f0bcd1443fa703cd68bded1c5e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:39.128981 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-7e36b4e5-6dce-41f8-a3d0-45433be1eab7 None None] MFA Rules not processed for user `8a14450faa014e1ebae3fe7f8e07d936`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:39.130416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7e36b4e5-6dce-41f8-a3d0-45433be1eab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:39.140720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0f4f475d-2025-439f-9922-217b4b38c830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '000d34f0bcd1443fa703cd68bded1c5e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:39.142209 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0f4f475d-2025-439f-9922-217b4b38c830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '000d34f0bcd1443fa703cd68bded1c5e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:39.227174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0f4f475d-2025-439f-9922-217b4b38c830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '000d34f0bcd1443fa703cd68bded1c5e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:39.243148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0f4f475d-2025-439f-9922-217b4b38c830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '000d34f0bcd1443fa703cd68bded1c5e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:39.256384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 858/1704] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:38 2022] DELETE /identity/v3/users/000d34f0bcd1443fa703cd68bded1c5e => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:39.260515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee194af-00d3-40f3-b8e0-37a82a150459 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:39.260850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee194af-00d3-40f3-b8e0-37a82a150459 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:39.261189 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee194af-00d3-40f3-b8e0-37a82a150459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:39.275449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 847/1705] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:39.279904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36b4e5-6dce-41f8-a3d0-45433be1eab7 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:39.281696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7e36b4e5-6dce-41f8-a3d0-45433be1eab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:39.305104 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-3ee194af-00d3-40f3-b8e0-37a82a150459 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:39.307752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3ee194af-00d3-40f3-b8e0-37a82a150459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:39.313403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36b4e5-6dce-41f8-a3d0-45433be1eab7 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:39.505315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9855b6-5fc9-461a-811e-2d262154f7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:39.505507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 859/1706] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:39.506249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9855b6-5fc9-461a-811e-2d262154f7b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:39.506574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9855b6-5fc9-461a-811e-2d262154f7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:39.506959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9855b6-5fc9-461a-811e-2d262154f7b4 admin admin] PATH_INFO: `/v3/projects/f4b487eadb984d0ba7962a5dfde2aedd/users/e62a6f741eee42f4a9138c6b26cf3d29/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:39.510786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3df7f1-dae0-49dc-afad-f87b93b2a999 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:39.511274 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3df7f1-dae0-49dc-afad-f87b93b2a999 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:39.511755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3df7f1-dae0-49dc-afad-f87b93b2a999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:39.530659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9855b6-5fc9-461a-811e-2d262154f7b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b487eadb984d0ba7962a5dfde2aedd, user_id=e62a6f741eee42f4a9138c6b26cf3d29, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:39.547298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ea3df7f1-dae0-49dc-afad-f87b93b2a999 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:39.549349 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ea3df7f1-dae0-49dc-afad-f87b93b2a999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:39.550006 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9855b6-5fc9-461a-811e-2d262154f7b4 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:39.561940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 860/1707] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:39.563231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3df7f1-dae0-49dc-afad-f87b93b2a999 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:39.565121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ea3df7f1-dae0-49dc-afad-f87b93b2a999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:39.583983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3df7f1-dae0-49dc-afad-f87b93b2a999 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:39.609444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 848/1708] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:39 2022] PUT /identity/v3/projects/f4b487eadb984d0ba7962a5dfde2aedd/users/e62a6f741eee42f4a9138c6b26cf3d29/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:39.609444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372babe-49f7-4603-9f6b-1a997db2b9c6 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:39.609444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372babe-49f7-4603-9f6b-1a997db2b9c6 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:39.609444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372babe-49f7-4603-9f6b-1a997db2b9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:39.710910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-0372babe-49f7-4603-9f6b-1a997db2b9c6 None None] MFA Rules not processed for user `284d7172360641a6b039d1bc23e1152e`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:39.713693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0372babe-49f7-4603-9f6b-1a997db2b9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:39.807031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168affbd-bcbf-499d-9a61-41002fdf1a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:39.810369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168affbd-bcbf-499d-9a61-41002fdf1a65 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:39.817396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168affbd-bcbf-499d-9a61-41002fdf1a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:39.817396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168affbd-bcbf-499d-9a61-41002fdf1a65 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:39.817396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168affbd-bcbf-499d-9a61-41002fdf1a65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:39.821675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168affbd-bcbf-499d-9a61-41002fdf1a65 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:39.892532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 861/1709] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:39 2022] POST /identity/v3/projects => generated 405 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:39.892532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168affbd-bcbf-499d-9a61-41002fdf1a65 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:39.905554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:39.919941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:39.942648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 849/1710] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:39.956149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372babe-49f7-4603-9f6b-1a997db2b9c6 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:39.960824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0372babe-49f7-4603-9f6b-1a997db2b9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:40.059867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372babe-49f7-4603-9f6b-1a997db2b9c6 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:40.073268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168affbd-bcbf-499d-9a61-41002fdf1a65 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:40.078606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80294fb4-a6f5-433f-a8af-d90cef9eea4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:40.079431 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80294fb4-a6f5-433f-a8af-d90cef9eea4b admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:40.079807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80294fb4-a6f5-433f-a8af-d90cef9eea4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:40.080222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80294fb4-a6f5-433f-a8af-d90cef9eea4b admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:40.081133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80294fb4-a6f5-433f-a8af-d90cef9eea4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:40.081397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80294fb4-a6f5-433f-a8af-d90cef9eea4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:40.084420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80294fb4-a6f5-433f-a8af-d90cef9eea4b admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:40.106662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 862/1711] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:40.108530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c563c4-4caa-4a7f-b2b2-c5f21c841e0e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:40.108993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c563c4-4caa-4a7f-b2b2-c5f21c841e0e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:40.109500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c563c4-4caa-4a7f-b2b2-c5f21c841e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:40.147795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-99c563c4-4caa-4a7f-b2b2-c5f21c841e0e None None] MFA Rules not processed for user `e62a6f741eee42f4a9138c6b26cf3d29`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:40.150309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-99c563c4-4caa-4a7f-b2b2-c5f21c841e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:40.216936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3828e37-7557-47e2-8602-fa25a5391cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:40.217798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3828e37-7557-47e2-8602-fa25a5391cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:40.218142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3828e37-7557-47e2-8602-fa25a5391cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:40.218739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3828e37-7557-47e2-8602-fa25a5391cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:40.239625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3828e37-7557-47e2-8602-fa25a5391cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:40.246528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3828e37-7557-47e2-8602-fa25a5391cf0 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:40.268889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 850/1712] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:40.270090 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3828e37-7557-47e2-8602-fa25a5391cf0 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:40.289162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3828e37-7557-47e2-8602-fa25a5391cf0 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:40.306091 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97809d85-6b0f-493c-9a8e-0c5e91d83046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:40.307026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97809d85-6b0f-493c-9a8e-0c5e91d83046 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:40.307435 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97809d85-6b0f-493c-9a8e-0c5e91d83046 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:40.307802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97809d85-6b0f-493c-9a8e-0c5e91d83046 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:40.310728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97809d85-6b0f-493c-9a8e-0c5e91d83046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:40.313905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97809d85-6b0f-493c-9a8e-0c5e91d83046 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:40.326267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 863/1713] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:40.328428 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3533925-6c57-4f80-b74a-7c2ffbc6d265 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:40.330350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3533925-6c57-4f80-b74a-7c2ffbc6d265 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:40.330725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3533925-6c57-4f80-b74a-7c2ffbc6d265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:40.366825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 851/1714] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:40 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:40.367890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97809d85-6b0f-493c-9a8e-0c5e91d83046 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:40.374280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-f3533925-6c57-4f80-b74a-7c2ffbc6d265 None None] MFA Rules not processed for user `284d7172360641a6b039d1bc23e1152e`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:40.378160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f3533925-6c57-4f80-b74a-7c2ffbc6d265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:40.392679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97809d85-6b0f-493c-9a8e-0c5e91d83046 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:40.406416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095b1d8-273a-45cd-b03d-cfe56ef19983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:40.408142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095b1d8-273a-45cd-b03d-cfe56ef19983 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:40.409247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095b1d8-273a-45cd-b03d-cfe56ef19983 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:40.410739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095b1d8-273a-45cd-b03d-cfe56ef19983 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:40.411720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095b1d8-273a-45cd-b03d-cfe56ef19983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:40.418437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095b1d8-273a-45cd-b03d-cfe56ef19983 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:40.486696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 852/1715] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:40 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:40.491805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095b1d8-273a-45cd-b03d-cfe56ef19983 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:40.497402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d095b1d8-273a-45cd-b03d-cfe56ef19983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:40.537187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 864/1716] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:40.539464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3533925-6c57-4f80-b74a-7c2ffbc6d265 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:40.569986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3533925-6c57-4f80-b74a-7c2ffbc6d265 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:40.647629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:40.686307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095b1d8-273a-45cd-b03d-cfe56ef19983 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:40.698676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:40.701913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:40.701913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:40.701913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] PATH_INFO: `/v3/users/bffae7676011443e8c95a0c6a66a2fc4` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:40.706307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=bffae7676011443e8c95a0c6a66a2fc4)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:40.715865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:40.738254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cbe8d-1612-4142-85cc-079c26397700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:40.738547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cbe8d-1612-4142-85cc-079c26397700 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:40.738914 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cbe8d-1612-4142-85cc-079c26397700 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:40.739196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cbe8d-1612-4142-85cc-079c26397700 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:40.741315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cbe8d-1612-4142-85cc-079c26397700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:40.746347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cbe8d-1612-4142-85cc-079c26397700 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:40.766498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:40.827880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 865/1717] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:40 2022] POST /identity/v3/projects => generated 393 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:40.837810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cbe8d-1612-4142-85cc-079c26397700 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:40.869911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bffae7676011443e8c95a0c6a66a2fc4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:40.889257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bffae7676011443e8c95a0c6a66a2fc4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:40.911946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bffae7676011443e8c95a0c6a66a2fc4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:40.920524 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bffae7676011443e8c95a0c6a66a2fc4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:40.952640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bffae7676011443e8c95a0c6a66a2fc4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:40.970799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 853/1718] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:40 2022] DELETE /identity/v3/users/bffae7676011443e8c95a0c6a66a2fc4 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:40.970799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:40.977185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:41.024141 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cbe8d-1612-4142-85cc-079c26397700 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:41.163974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af35bd-9adc-4773-96e7-382b66892edf admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:41.171738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582e10f-dce9-4d16-9f65-1e85a0886f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:41.174576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582e10f-dce9-4d16-9f65-1e85a0886f02 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:41.175558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582e10f-dce9-4d16-9f65-1e85a0886f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:41.176564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582e10f-dce9-4d16-9f65-1e85a0886f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:41.177869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582e10f-dce9-4d16-9f65-1e85a0886f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:41.181274 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582e10f-dce9-4d16-9f65-1e85a0886f02 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:41.190523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea27ba1-4b1b-40ca-9ac2-efb855b5c93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:41.192146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea27ba1-4b1b-40ca-9ac2-efb855b5c93e admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:41.192715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea27ba1-4b1b-40ca-9ac2-efb855b5c93e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:41.193170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea27ba1-4b1b-40ca-9ac2-efb855b5c93e admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:41.194060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea27ba1-4b1b-40ca-9ac2-efb855b5c93e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:41.199269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea27ba1-4b1b-40ca-9ac2-efb855b5c93e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:41.229091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 866/1719] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:41.230382 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582e10f-dce9-4d16-9f65-1e85a0886f02 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:41.232655 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9582e10f-dce9-4d16-9f65-1e85a0886f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:41.256738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 854/1720] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:40 2022] POST /identity/v3/users => generated 286 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:41.258243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea27ba1-4b1b-40ca-9ac2-efb855b5c93e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:41.260793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0ea27ba1-4b1b-40ca-9ac2-efb855b5c93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:41.281725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea27ba1-4b1b-40ca-9ac2-efb855b5c93e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:41.310044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1100d5-ebe0-4b4c-88bf-7959834c0847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:41.311671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1100d5-ebe0-4b4c-88bf-7959834c0847 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:41.312691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1100d5-ebe0-4b4c-88bf-7959834c0847 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:41.315708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1100d5-ebe0-4b4c-88bf-7959834c0847 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:41.317105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1100d5-ebe0-4b4c-88bf-7959834c0847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:41.325357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1100d5-ebe0-4b4c-88bf-7959834c0847 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:41.400384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:41.411300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 855/1721] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:41 2022] POST /identity/v3/users => generated 303 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:41.428670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d535f7-46b0-47f8-8338-92c99927991b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:41.429896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d535f7-46b0-47f8-8338-92c99927991b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:41.432016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582e10f-dce9-4d16-9f65-1e85a0886f02 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:41.432422 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d535f7-46b0-47f8-8338-92c99927991b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:41.448937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac15b8-5908-4f49-9e9c-5432fad25b53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:41.451067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac15b8-5908-4f49-9e9c-5432fad25b53 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:41.451842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac15b8-5908-4f49-9e9c-5432fad25b53 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:41.452921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac15b8-5908-4f49-9e9c-5432fad25b53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:41.456552 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fcac15b8-5908-4f49-9e9c-5432fad25b53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:41.512782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-47d535f7-46b0-47f8-8338-92c99927991b None None] MFA Rules not processed for user `d4c6ceff33ee449193fe6d4f7b2c01ec`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:41.517717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-47d535f7-46b0-47f8-8338-92c99927991b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:41.604074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac15b8-5908-4f49-9e9c-5432fad25b53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:41.607399 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac15b8-5908-4f49-9e9c-5432fad25b53 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:41.669933 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 856/1722] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:41.672258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157acd2f-ea53-435e-a0c4-4a13d11da76e None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:41.673909 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157acd2f-ea53-435e-a0c4-4a13d11da76e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:41.674635 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157acd2f-ea53-435e-a0c4-4a13d11da76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:41.683229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 867/1723] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:41.687879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac15b8-5908-4f49-9e9c-5432fad25b53 service nova] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:41.695732 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fcac15b8-5908-4f49-9e9c-5432fad25b53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:41.708163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-157acd2f-ea53-435e-a0c4-4a13d11da76e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:41.710408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-157acd2f-ea53-435e-a0c4-4a13d11da76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:41.758523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 857/1724] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:41.759429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac15b8-5908-4f49-9e9c-5432fad25b53 service nova] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:41.761878 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157acd2f-ea53-435e-a0c4-4a13d11da76e None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:41.780980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077378bf-95ca-4f6c-abd9-39e90a88a4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:41.783278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077378bf-95ca-4f6c-abd9-39e90a88a4bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:41.783588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077378bf-95ca-4f6c-abd9-39e90a88a4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:41.785866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077378bf-95ca-4f6c-abd9-39e90a88a4bd admin admin] PATH_INFO: `/v3/projects/ede33262046e4f2e937fd47701417164/users/419f34f913c449afab926ef39471805a/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:41.786767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:DELETE=9,keystone:SELECT=84,keystone:INSERT=7 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:41.800905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157acd2f-ea53-435e-a0c4-4a13d11da76e None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:41.819138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c6219-23c0-4f37-9ca2-33ec3990bdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:41.819900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c6219-23c0-4f37-9ca2-33ec3990bdf9 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:41.820189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c6219-23c0-4f37-9ca2-33ec3990bdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:41.820540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c6219-23c0-4f37-9ca2-33ec3990bdf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:41.821699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c6219-23c0-4f37-9ca2-33ec3990bdf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:41.823943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c6219-23c0-4f37-9ca2-33ec3990bdf9 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:41.838172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077378bf-95ca-4f6c-abd9-39e90a88a4bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede33262046e4f2e937fd47701417164, user_id=419f34f913c449afab926ef39471805a, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:41.843307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077378bf-95ca-4f6c-abd9-39e90a88a4bd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:41.857796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 858/1725] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:41.857796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c6219-23c0-4f37-9ca2-33ec3990bdf9 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:41.877885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 868/1726] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:41 2022] PUT /identity/v3/projects/ede33262046e4f2e937fd47701417164/users/419f34f913c449afab926ef39471805a/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:41.879442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077378bf-95ca-4f6c-abd9-39e90a88a4bd admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:41.886601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-077378bf-95ca-4f6c-abd9-39e90a88a4bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:41.890407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c6219-23c0-4f37-9ca2-33ec3990bdf9 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:42.085525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077378bf-95ca-4f6c-abd9-39e90a88a4bd admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:42.103812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e202d-0fa0-4ecc-a156-8eb148c29154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:42.105094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e202d-0fa0-4ecc-a156-8eb148c29154 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:42.105632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e202d-0fa0-4ecc-a156-8eb148c29154 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:42.108957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e202d-0fa0-4ecc-a156-8eb148c29154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:42.215717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b656b-95d6-41e5-b3af-5d2023c34c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:42.217108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b656b-95d6-41e5-b3af-5d2023c34c2a admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:42.217536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b656b-95d6-41e5-b3af-5d2023c34c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:42.217951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b656b-95d6-41e5-b3af-5d2023c34c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:42.218885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b656b-95d6-41e5-b3af-5d2023c34c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:42.221699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b656b-95d6-41e5-b3af-5d2023c34c2a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:42.293625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 859/1727] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:42.294942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b656b-95d6-41e5-b3af-5d2023c34c2a admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:42.341860 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b656b-95d6-41e5-b3af-5d2023c34c2a admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:42.356433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9dcac-34e3-4b88-9033-aa8a0586ce94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:42.361932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9dcac-34e3-4b88-9033-aa8a0586ce94 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:42.362464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9dcac-34e3-4b88-9033-aa8a0586ce94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:42.362956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9dcac-34e3-4b88-9033-aa8a0586ce94 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:42.363921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9dcac-34e3-4b88-9033-aa8a0586ce94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:42.365744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9dcac-34e3-4b88-9033-aa8a0586ce94 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:42.395415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 860/1728] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:42.403695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9dcac-34e3-4b88-9033-aa8a0586ce94 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:42.431844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9dcac-34e3-4b88-9033-aa8a0586ce94 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:42.444209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43ab70-a862-4f1f-b49c-995b8c60169c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:42.445316 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43ab70-a862-4f1f-b49c-995b8c60169c admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:42.445919 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43ab70-a862-4f1f-b49c-995b8c60169c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:42.446320 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43ab70-a862-4f1f-b49c-995b8c60169c admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:42.447534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43ab70-a862-4f1f-b49c-995b8c60169c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:42.451773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43ab70-a862-4f1f-b49c-995b8c60169c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:42.484686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 861/1729] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:42.489155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43ab70-a862-4f1f-b49c-995b8c60169c admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:42.491487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4d43ab70-a862-4f1f-b49c-995b8c60169c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:42.519771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43ab70-a862-4f1f-b49c-995b8c60169c admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:42.537814 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e202d-0fa0-4ecc-a156-8eb148c29154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:42.546643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e202d-0fa0-4ecc-a156-8eb148c29154 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:42.549321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:42.551405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:42.552069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:42.556558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] PATH_INFO: `/v3/projects/46aced1136734ec4b31804fb64155e88` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:42.561499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46aced1136734ec4b31804fb64155e88)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:42.573115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:42.578466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 869/1730] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:42.583191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e202d-0fa0-4ecc-a156-8eb148c29154 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:42.602742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e202d-0fa0-4ecc-a156-8eb148c29154 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:42.623246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c41dc-0d12-493e-a917-e642bca7eb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:42.624434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c41dc-0d12-493e-a917-e642bca7eb8e admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:42.624827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c41dc-0d12-493e-a917-e642bca7eb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:42.625208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c41dc-0d12-493e-a917-e642bca7eb8e admin admin] PATH_INFO: `/v3/projects/0da74ec711dd4d2e85e8f970f4c5a93e/users/33aa784807234b42a3984dcd47e463a0/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:42.649112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c41dc-0d12-493e-a917-e642bca7eb8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da74ec711dd4d2e85e8f970f4c5a93e, user_id=33aa784807234b42a3984dcd47e463a0, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:42.653062 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c41dc-0d12-493e-a917-e642bca7eb8e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:42.668569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:42.682148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 870/1731] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:42 2022] PUT /identity/v3/projects/0da74ec711dd4d2e85e8f970f4c5a93e/users/33aa784807234b42a3984dcd47e463a0/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:42.683529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c41dc-0d12-493e-a917-e642bca7eb8e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:42.770926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46aced1136734ec4b31804fb64155e88', 'request_id': 'req-17ef76e3-5f5f-4a66-8e06-6731535c9590'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:42.785658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46aced1136734ec4b31804fb64155e88', 'request_id': 'req-17ef76e3-5f5f-4a66-8e06-6731535c9590'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:42.845378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46aced1136734ec4b31804fb64155e88', 'request_id': 'req-17ef76e3-5f5f-4a66-8e06-6731535c9590'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:42.885468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] The token cache is being invalidate because project 46aced1136734ec4b31804fb64155e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:42.886036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:42.887819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17ef76e3-5f5f-4a66-8e06-6731535c9590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:42.891146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 862/1732] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:42 2022] DELETE /identity/v3/projects/46aced1136734ec4b31804fb64155e88 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:42.894955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c68c7-68f7-4269-ad8a-6fc16c282a8d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:42.895255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c68c7-68f7-4269-ad8a-6fc16c282a8d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:42.895540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c68c7-68f7-4269-ad8a-6fc16c282a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:42.925829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-0f4c68c7-68f7-4269-ad8a-6fc16c282a8d None None] MFA Rules not processed for user `1f026e791bf24a9fab1aaedb794e6229`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:42.927153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0f4c68c7-68f7-4269-ad8a-6fc16c282a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:42.955613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c41dc-0d12-493e-a917-e642bca7eb8e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:43.060937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:43.116122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 863/1733] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:43.120974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c68c7-68f7-4269-ad8a-6fc16c282a8d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:43.128230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0f4c68c7-68f7-4269-ad8a-6fc16c282a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:43.141150 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25203d4e-2934-438b-9675-531416e64635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:43.143277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203d4e-2934-438b-9675-531416e64635 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:43.147562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203d4e-2934-438b-9675-531416e64635 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:43.147907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203d4e-2934-438b-9675-531416e64635 admin admin] PATH_INFO: `/v3/projects/78fe5b511fce440799085a216415371a/users/e31ac10534c74888ae6a9f71b840af1c/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:43.245459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25203d4e-2934-438b-9675-531416e64635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78fe5b511fce440799085a216415371a, user_id=e31ac10534c74888ae6a9f71b840af1c, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:43.252368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25203d4e-2934-438b-9675-531416e64635 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:43.307707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 871/1734] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:42 2022] PUT /identity/v3/projects/78fe5b511fce440799085a216415371a/users/e31ac10534c74888ae6a9f71b840af1c/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:43.307707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25203d4e-2934-438b-9675-531416e64635 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:43.321987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-25203d4e-2934-438b-9675-531416e64635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:43.430256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c68c7-68f7-4269-ad8a-6fc16c282a8d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:43.472885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25203d4e-2934-438b-9675-531416e64635 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:43.484818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db99fa-6af4-4e0d-8f04-d304d0c21ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:43.486302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db99fa-6af4-4e0d-8f04-d304d0c21ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:43.486656 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db99fa-6af4-4e0d-8f04-d304d0c21ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:43.487000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db99fa-6af4-4e0d-8f04-d304d0c21ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:43.487821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db99fa-6af4-4e0d-8f04-d304d0c21ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:43.494881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db99fa-6af4-4e0d-8f04-d304d0c21ac6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:43.511810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 872/1735] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:43.513757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cb500-fd90-446a-925b-3aef12010657 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:43.514118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cb500-fd90-446a-925b-3aef12010657 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:43.514550 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cb500-fd90-446a-925b-3aef12010657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:43.558516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ade25-8b40-457c-9711-9f45a39a2e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:43.560371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ade25-8b40-457c-9711-9f45a39a2e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:43.561128 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ade25-8b40-457c-9711-9f45a39a2e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:43.561551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ade25-8b40-457c-9711-9f45a39a2e5f admin admin] PATH_INFO: `/v3/projects/ede33262046e4f2e937fd47701417164/users/419f34f913c449afab926ef39471805a/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:43.564631 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ade25-8b40-457c-9711-9f45a39a2e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede33262046e4f2e937fd47701417164, user_id=419f34f913c449afab926ef39471805a, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:43.568679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ade25-8b40-457c-9711-9f45a39a2e5f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:43.568815 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-4b0cb500-fd90-446a-925b-3aef12010657 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:43.582295 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 864/1736] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:43 2022] PUT /identity/v3/projects/ede33262046e4f2e937fd47701417164/users/419f34f913c449afab926ef39471805a/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:43.583904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c6d52-07aa-4ac2-8e1f-363dfdb65629 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:43.584363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c6d52-07aa-4ac2-8e1f-363dfdb65629 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:43.584775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c6d52-07aa-4ac2-8e1f-363dfdb65629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:43.599766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4b0cb500-fd90-446a-925b-3aef12010657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:43.638101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-2f2c6d52-07aa-4ac2-8e1f-363dfdb65629 None None] MFA Rules not processed for user `e31ac10534c74888ae6a9f71b840af1c`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:43.639947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2f2c6d52-07aa-4ac2-8e1f-363dfdb65629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:43.677536 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:43.778593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 865/1737] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:43.780813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34ce03-a742-4844-a9d8-5fa4fc1a6864 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:43.780813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34ce03-a742-4844-a9d8-5fa4fc1a6864 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:43.781102 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34ce03-a742-4844-a9d8-5fa4fc1a6864 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:43.782646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 866/1738] 10.222.0.85 () {54 vars in 912 bytes} [Tue Jul 19 20:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:54:43.789333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34ce03-a742-4844-a9d8-5fa4fc1a6864 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:43.826224 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 873/1739] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:43.828573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cb500-fd90-446a-925b-3aef12010657 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:43.833608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4b0cb500-fd90-446a-925b-3aef12010657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:43.861604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:43.896543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cb500-fd90-446a-925b-3aef12010657 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:43.920096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c83024-2623-474c-afc0-67e57e2b414e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:43.920493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34ce03-a742-4844-a9d8-5fa4fc1a6864 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:43.922479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c83024-2623-474c-afc0-67e57e2b414e admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:43.923092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c83024-2623-474c-afc0-67e57e2b414e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:43.923534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c83024-2623-474c-afc0-67e57e2b414e admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:43.925597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c83024-2623-474c-afc0-67e57e2b414e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:43.928649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c83024-2623-474c-afc0-67e57e2b414e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:43.936078 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4eea9-cb26-4b21-a051-0687e0b8eea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:43.937240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4eea9-cb26-4b21-a051-0687e0b8eea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:43.937565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4eea9-cb26-4b21-a051-0687e0b8eea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:43.937849 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4eea9-cb26-4b21-a051-0687e0b8eea7 admin admin] PATH_INFO: `/v3/projects/0da74ec711dd4d2e85e8f970f4c5a93e/users/33aa784807234b42a3984dcd47e463a0/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:43.947468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b4eea9-cb26-4b21-a051-0687e0b8eea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da74ec711dd4d2e85e8f970f4c5a93e, user_id=33aa784807234b42a3984dcd47e463a0, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:43.950453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b4eea9-cb26-4b21-a051-0687e0b8eea7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:43.983067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 867/1740] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:43 2022] PUT /identity/v3/projects/0da74ec711dd4d2e85e8f970f4c5a93e/users/33aa784807234b42a3984dcd47e463a0/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:43.988093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d89ce-18b3-4ab1-bc86-3823b5099c8d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:43.988972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d89ce-18b3-4ab1-bc86-3823b5099c8d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:43.988972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d89ce-18b3-4ab1-bc86-3823b5099c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:44.027336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 874/1741] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:43 2022] POST /identity/v3/projects => generated 371 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:44.027336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59c749-386c-4391-82b1-80fffca86646 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:44.027336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59c749-386c-4391-82b1-80fffca86646 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:44.027914 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59c749-386c-4391-82b1-80fffca86646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:44.060924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-bf59c749-386c-4391-82b1-80fffca86646 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:44.062864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-bf59c749-386c-4391-82b1-80fffca86646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:44.156169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-756d89ce-18b3-4ab1-bc86-3823b5099c8d None None] MFA Rules not processed for user `5db827b3a66843749e04376559aa27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:44.161508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-756d89ce-18b3-4ab1-bc86-3823b5099c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:44.323161 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 868/1742] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:44.328688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162d020-7515-447d-993e-1b4078e5e17e None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:44.329207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162d020-7515-447d-993e-1b4078e5e17e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:44.329532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162d020-7515-447d-993e-1b4078e5e17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:44.365478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 875/1743] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:44.366970 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59c749-386c-4391-82b1-80fffca86646 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:44.396518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3162d020-7515-447d-993e-1b4078e5e17e None None] MFA Rules not processed for user `33aa784807234b42a3984dcd47e463a0`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:44.397808 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3162d020-7515-447d-993e-1b4078e5e17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:44.417017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59c749-386c-4391-82b1-80fffca86646 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:44.451387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be559d57-5a85-406c-9fd3-aff7b2fa67f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:44.452315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be559d57-5a85-406c-9fd3-aff7b2fa67f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:44.452824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be559d57-5a85-406c-9fd3-aff7b2fa67f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:44.453227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be559d57-5a85-406c-9fd3-aff7b2fa67f5 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:44.454142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be559d57-5a85-406c-9fd3-aff7b2fa67f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:44.457000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be559d57-5a85-406c-9fd3-aff7b2fa67f5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:44.494191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 876/1744] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:44 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:44.498529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba1119-31c1-4681-8d0b-9828f65bcd30 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:44.501348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba1119-31c1-4681-8d0b-9828f65bcd30 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:44.501790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba1119-31c1-4681-8d0b-9828f65bcd30 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:44.503690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 877/1745] 10.222.0.85 () {54 vars in 912 bytes} [Tue Jul 19 20:54:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:54:44.509711 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba1119-31c1-4681-8d0b-9828f65bcd30 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:44.512026 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-abba1119-31c1-4681-8d0b-9828f65bcd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:44.527746 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 869/1746] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:44.527746 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162d020-7515-447d-993e-1b4078e5e17e None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:44.531104 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba1119-31c1-4681-8d0b-9828f65bcd30 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:44.540927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f345a-552c-49e8-adf9-1202d83a3938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:44.541832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f345a-552c-49e8-adf9-1202d83a3938 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:44.541832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f345a-552c-49e8-adf9-1202d83a3938 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:44.542614 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f345a-552c-49e8-adf9-1202d83a3938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:44.551892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162d020-7515-447d-993e-1b4078e5e17e None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:44.559331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f345a-552c-49e8-adf9-1202d83a3938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:44.562283 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f345a-552c-49e8-adf9-1202d83a3938 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:44.562917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532f639-a589-4877-a29b-4eb462cc2fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:44.564603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532f639-a589-4877-a29b-4eb462cc2fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:44.564957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532f639-a589-4877-a29b-4eb462cc2fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:44.565340 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532f639-a589-4877-a29b-4eb462cc2fc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:44.566091 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532f639-a589-4877-a29b-4eb462cc2fc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:44.566417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532f639-a589-4877-a29b-4eb462cc2fc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:44.568737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532f639-a589-4877-a29b-4eb462cc2fc6 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:44.586373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 878/1747] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:44.587411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f345a-552c-49e8-adf9-1202d83a3938 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:44.592237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 870/1748] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:44.593331 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e906be-b2b0-4a6c-83b1-0917dfb4a9ca None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:44.593625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e906be-b2b0-4a6c-83b1-0917dfb4a9ca None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:44.593910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e906be-b2b0-4a6c-83b1-0917dfb4a9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:44.619575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c6e906be-b2b0-4a6c-83b1-0917dfb4a9ca None None] MFA Rules not processed for user `5db827b3a66843749e04376559aa27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:44.621955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c6e906be-b2b0-4a6c-83b1-0917dfb4a9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:44.634272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 871/1749] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:44.635827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e906be-b2b0-4a6c-83b1-0917dfb4a9ca None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:44.637706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c6e906be-b2b0-4a6c-83b1-0917dfb4a9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:44.648515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f345a-552c-49e8-adf9-1202d83a3938 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:44.671240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fc1d4-8822-4ca7-b868-cbb7862888ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:44.673400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fc1d4-8822-4ca7-b868-cbb7862888ab admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:44.673400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fc1d4-8822-4ca7-b868-cbb7862888ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:44.673729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fc1d4-8822-4ca7-b868-cbb7862888ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:44.674894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fc1d4-8822-4ca7-b868-cbb7862888ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:44.675173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:44.677685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fc1d4-8822-4ca7-b868-cbb7862888ab admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:44.740454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 879/1750] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:44.740759 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e906be-b2b0-4a6c-83b1-0917dfb4a9ca None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:44.746384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fc1d4-8822-4ca7-b868-cbb7862888ab admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:44.747589 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf21e80-af8a-43d4-b278-cc4b90a41eca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:44.748487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf21e80-af8a-43d4-b278-cc4b90a41eca service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:44.749323 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf21e80-af8a-43d4-b278-cc4b90a41eca service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:44.749680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf21e80-af8a-43d4-b278-cc4b90a41eca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:44.788809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fc1d4-8822-4ca7-b868-cbb7862888ab admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:44.804016 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf21e80-af8a-43d4-b278-cc4b90a41eca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:44.812978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22575d65-a1fb-4bac-a7df-0edaad6985a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:44.814036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575d65-a1fb-4bac-a7df-0edaad6985a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:44.819377 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575d65-a1fb-4bac-a7df-0edaad6985a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:44.819377 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575d65-a1fb-4bac-a7df-0edaad6985a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:44.819377 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22575d65-a1fb-4bac-a7df-0edaad6985a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:44.819377 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22575d65-a1fb-4bac-a7df-0edaad6985a5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:44.822448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf21e80-af8a-43d4-b278-cc4b90a41eca service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:44.854927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 872/1751] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:44.859761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf21e80-af8a-43d4-b278-cc4b90a41eca service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:44.903594 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 880/1752] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:44 2022] POST /identity/v3/projects => generated 355 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:44.904674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22575d65-a1fb-4bac-a7df-0edaad6985a5 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:44.930434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf21e80-af8a-43d4-b278-cc4b90a41eca service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:45.073838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe763941-0d07-4020-b8e1-8e22ef0a0f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:45.076633 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe763941-0d07-4020-b8e1-8e22ef0a0f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:45.077663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe763941-0d07-4020-b8e1-8e22ef0a0f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:45.078131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe763941-0d07-4020-b8e1-8e22ef0a0f19 admin admin] PATH_INFO: `/v3/projects/4a853ed2954f407abe423880763da34a/users/ad3bc9cfff0e4797906a0c0a496ea484/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:45.098454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:45.112420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe763941-0d07-4020-b8e1-8e22ef0a0f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a853ed2954f407abe423880763da34a, user_id=ad3bc9cfff0e4797906a0c0a496ea484, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:45.115441 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe763941-0d07-4020-b8e1-8e22ef0a0f19 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:45.134802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22575d65-a1fb-4bac-a7df-0edaad6985a5 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:45.151931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 873/1753] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:44 2022] PUT /identity/v3/projects/4a853ed2954f407abe423880763da34a/users/ad3bc9cfff0e4797906a0c0a496ea484/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:45.155322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe763941-0d07-4020-b8e1-8e22ef0a0f19 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:45.303486 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90a8a1-f44f-4a40-887e-39c6297b3c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:45.305600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90a8a1-f44f-4a40-887e-39c6297b3c40 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:45.306105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90a8a1-f44f-4a40-887e-39c6297b3c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:45.306901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90a8a1-f44f-4a40-887e-39c6297b3c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:45.335470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90a8a1-f44f-4a40-887e-39c6297b3c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:45.338629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90a8a1-f44f-4a40-887e-39c6297b3c40 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:45.365026 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 881/1754] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:45.369245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90a8a1-f44f-4a40-887e-39c6297b3c40 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:45.372892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9e90a8a1-f44f-4a40-887e-39c6297b3c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:45.392495 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe763941-0d07-4020-b8e1-8e22ef0a0f19 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:45.413695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90a8a1-f44f-4a40-887e-39c6297b3c40 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:45.417996 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3377d-2d19-4432-82eb-bc1593e5f8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:45.418780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3377d-2d19-4432-82eb-bc1593e5f8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:45.419310 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3377d-2d19-4432-82eb-bc1593e5f8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:45.419660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3377d-2d19-4432-82eb-bc1593e5f8d5 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:45.420848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3377d-2d19-4432-82eb-bc1593e5f8d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:45.422614 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:45.425788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3377d-2d19-4432-82eb-bc1593e5f8d5 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:45.426295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:45.427073 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:45.427554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] PATH_INFO: `/v3/projects/785f77170a324c33833523407884657f` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:45.429945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=785f77170a324c33833523407884657f)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:45.435386 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:45.470018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 874/1755] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:45 2022] POST /identity/v3/users => generated 278 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:45.471130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3377d-2d19-4432-82eb-bc1593e5f8d5 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:45.477175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-5df3377d-2d19-4432-82eb-bc1593e5f8d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:45.605239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '785f77170a324c33833523407884657f', 'request_id': 'req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:45.618281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:45.633174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '785f77170a324c33833523407884657f', 'request_id': 'req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:45.643961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '785f77170a324c33833523407884657f', 'request_id': 'req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:45.688053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3377d-2d19-4432-82eb-bc1593e5f8d5 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:45.691134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] The token cache is being invalidate because project 785f77170a324c33833523407884657f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:45.699101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:45.704046 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:45.708980 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 882/1756] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:45 2022] DELETE /identity/v3/projects/785f77170a324c33833523407884657f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:45.710854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:45.718276 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:45.884060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b45e8b-1359-4760-a152-09a3d1db9a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:45.885486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b45e8b-1359-4760-a152-09a3d1db9a64 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:45.886184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b45e8b-1359-4760-a152-09a3d1db9a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:45.894155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b45e8b-1359-4760-a152-09a3d1db9a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:45.904522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f3b45e8b-1359-4760-a152-09a3d1db9a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:45.975520 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4c5c1-6f74-4d9e-9b79-31795297b7d5 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:46.008178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781c182-5de3-4e27-8e7d-eb16dbd62003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:46.009438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781c182-5de3-4e27-8e7d-eb16dbd62003 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:46.009963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781c182-5de3-4e27-8e7d-eb16dbd62003 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:46.010487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781c182-5de3-4e27-8e7d-eb16dbd62003 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:46.011985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781c182-5de3-4e27-8e7d-eb16dbd62003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:46.015251 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781c182-5de3-4e27-8e7d-eb16dbd62003 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:46.065236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 883/1757] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:46.067004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781c182-5de3-4e27-8e7d-eb16dbd62003 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:46.070265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a781c182-5de3-4e27-8e7d-eb16dbd62003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:46.114420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781c182-5de3-4e27-8e7d-eb16dbd62003 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:46.116775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b45e8b-1359-4760-a152-09a3d1db9a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:46.118307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b45e8b-1359-4760-a152-09a3d1db9a64 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:46.139571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461938b-84a9-4c3c-b599-a5c040d2f6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:46.142771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461938b-84a9-4c3c-b599-a5c040d2f6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:46.144480 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 875/1758] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:46.146276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b45e8b-1359-4760-a152-09a3d1db9a64 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:46.148171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461938b-84a9-4c3c-b599-a5c040d2f6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:46.148571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f3b45e8b-1359-4760-a152-09a3d1db9a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:46.148744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461938b-84a9-4c3c-b599-a5c040d2f6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:46.149429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461938b-84a9-4c3c-b599-a5c040d2f6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:46.153901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461938b-84a9-4c3c-b599-a5c040d2f6f3 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:46.183825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b45e8b-1359-4760-a152-09a3d1db9a64 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:46.185541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 884/1759] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:46.186352 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461938b-84a9-4c3c-b599-a5c040d2f6f3 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:46.212318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f814a35-fbeb-46ef-abaf-b503242afbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:46.213916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814a35-fbeb-46ef-abaf-b503242afbfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:46.214571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814a35-fbeb-46ef-abaf-b503242afbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:46.215029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814a35-fbeb-46ef-abaf-b503242afbfd admin admin] PATH_INFO: `/v3/users/60cdf320996d45a8bf9d9623044ad0aa` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:46.218283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f814a35-fbeb-46ef-abaf-b503242afbfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=60cdf320996d45a8bf9d9623044ad0aa)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:46.223618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461938b-84a9-4c3c-b599-a5c040d2f6f3 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:46.238396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6fa51-56bd-4009-9d26-abaff1cb606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:46.239561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6fa51-56bd-4009-9d26-abaff1cb606d admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:46.239699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f814a35-fbeb-46ef-abaf-b503242afbfd admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:46.239831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6fa51-56bd-4009-9d26-abaff1cb606d admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:46.240298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6fa51-56bd-4009-9d26-abaff1cb606d admin admin] PATH_INFO: `/v3/projects/4a853ed2954f407abe423880763da34a/users/ad3bc9cfff0e4797906a0c0a496ea484/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:46.247485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae6fa51-56bd-4009-9d26-abaff1cb606d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a853ed2954f407abe423880763da34a, user_id=ad3bc9cfff0e4797906a0c0a496ea484, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:46.256867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae6fa51-56bd-4009-9d26-abaff1cb606d admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:46.267211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:46.296522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 885/1760] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:46 2022] PUT /identity/v3/projects/4a853ed2954f407abe423880763da34a/users/ad3bc9cfff0e4797906a0c0a496ea484/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:46.298090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2568b6a-aa2a-4934-9587-551e3395f1ed None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:46.298390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2568b6a-aa2a-4934-9587-551e3395f1ed None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:46.298681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2568b6a-aa2a-4934-9587-551e3395f1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:46.383011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-f2568b6a-aa2a-4934-9587-551e3395f1ed None None] MFA Rules not processed for user `419f34f913c449afab926ef39471805a`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:46.384247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f2568b6a-aa2a-4934-9587-551e3395f1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:46.420914 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7f814a35-fbeb-46ef-abaf-b503242afbfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60cdf320996d45a8bf9d9623044ad0aa'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:46.466180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7f814a35-fbeb-46ef-abaf-b503242afbfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60cdf320996d45a8bf9d9623044ad0aa'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:46.506770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7f814a35-fbeb-46ef-abaf-b503242afbfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60cdf320996d45a8bf9d9623044ad0aa'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:46.510895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7f814a35-fbeb-46ef-abaf-b503242afbfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60cdf320996d45a8bf9d9623044ad0aa'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:46.572253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 886/1761] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:46 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:46.577917 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2568b6a-aa2a-4934-9587-551e3395f1ed None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:46.580382 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f2568b6a-aa2a-4934-9587-551e3395f1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:46.592581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7f814a35-fbeb-46ef-abaf-b503242afbfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60cdf320996d45a8bf9d9623044ad0aa'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:46.630187 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 876/1762] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:46 2022] DELETE /identity/v3/users/60cdf320996d45a8bf9d9623044ad0aa => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:46.632694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04c84-9f03-459b-83c5-b7f27fe96455 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:46.633267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04c84-9f03-459b-83c5-b7f27fe96455 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:46.633662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04c84-9f03-459b-83c5-b7f27fe96455 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:46.650360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 877/1763] 10.222.0.85 () {54 vars in 961 bytes} [Tue Jul 19 20:54:46 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:54:46.659652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877aa99-48df-4111-841b-9b0f9008aa3d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:46.660225 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877aa99-48df-4111-841b-9b0f9008aa3d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:46.660732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877aa99-48df-4111-841b-9b0f9008aa3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:46.770877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c877aa99-48df-4111-841b-9b0f9008aa3d None None] MFA Rules not processed for user `ad3bc9cfff0e4797906a0c0a496ea484`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:46.784791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c877aa99-48df-4111-841b-9b0f9008aa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:46.838892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:46.901689 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2568b6a-aa2a-4934-9587-551e3395f1ed None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:47.045727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6ada4-a527-4728-8c1e-27be52df4289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:47.048253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6ada4-a527-4728-8c1e-27be52df4289 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:47.051626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6ada4-a527-4728-8c1e-27be52df4289 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:47.051920 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6ada4-a527-4728-8c1e-27be52df4289 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:47.054788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6ada4-a527-4728-8c1e-27be52df4289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:47.058960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6ada4-a527-4728-8c1e-27be52df4289 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:47.092560 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 887/1764] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 520 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:47.094185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08532282-a50f-4ba5-b2aa-ff84438e9f41 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:47.097172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08532282-a50f-4ba5-b2aa-ff84438e9f41 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:47.097629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08532282-a50f-4ba5-b2aa-ff84438e9f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:47.142664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-08532282-a50f-4ba5-b2aa-ff84438e9f41 None None] MFA Rules not processed for user `fffebfc51dba47f68263c0bcd6b18d22`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:47.144396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-08532282-a50f-4ba5-b2aa-ff84438e9f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:47.162542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 878/1765] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:46 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:47.162542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877aa99-48df-4111-841b-9b0f9008aa3d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:47.173653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c877aa99-48df-4111-841b-9b0f9008aa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:47.187685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:47.304198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 888/1766] 10.222.0.85 () {58 vars in 1058 bytes} [Tue Jul 19 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:47.306696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08532282-a50f-4ba5-b2aa-ff84438e9f41 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:47.327239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08532282-a50f-4ba5-b2aa-ff84438e9f41 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:47.365824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d062b-eded-4cd8-87c9-3df653f26931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:47.367303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d062b-eded-4cd8-87c9-3df653f26931 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:47.369168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d062b-eded-4cd8-87c9-3df653f26931 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:47.371327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d062b-eded-4cd8-87c9-3df653f26931 admin admin] PATH_INFO: `/v3/projects/092d039a2b1c4b70819b1f3aa968ec92/users/850e86e60d704092a1bb5a96db587196/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:47.383699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877aa99-48df-4111-841b-9b0f9008aa3d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:47.393550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc409-3545-4774-b2b7-5d9fa632a222 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:47.394515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc409-3545-4774-b2b7-5d9fa632a222 service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:47.394811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc409-3545-4774-b2b7-5d9fa632a222 service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:47.395115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc409-3545-4774-b2b7-5d9fa632a222 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:47.397273 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e7edc409-3545-4774-b2b7-5d9fa632a222 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:47.412525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d062b-eded-4cd8-87c9-3df653f26931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=092d039a2b1c4b70819b1f3aa968ec92, user_id=850e86e60d704092a1bb5a96db587196, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:47.412777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edc409-3545-4774-b2b7-5d9fa632a222 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:47.414477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edc409-3545-4774-b2b7-5d9fa632a222 service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:47.415901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d062b-eded-4cd8-87c9-3df653f26931 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:47.435962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 879/1767] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:47.437272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 889/1768] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:47 2022] PUT /identity/v3/projects/092d039a2b1c4b70819b1f3aa968ec92/users/850e86e60d704092a1bb5a96db587196/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:47.438952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d062b-eded-4cd8-87c9-3df653f26931 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:47.453717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc409-3545-4774-b2b7-5d9fa632a222 service placement] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:47.457521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e7edc409-3545-4774-b2b7-5d9fa632a222 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:47.634929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d062b-eded-4cd8-87c9-3df653f26931 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:47.644520 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71591aa-f74a-4f29-b9d0-00fd2203eb1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:47.645870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc409-3545-4774-b2b7-5d9fa632a222 service placement] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:47.646659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71591aa-f74a-4f29-b9d0-00fd2203eb1b service placement] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:47.646659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71591aa-f74a-4f29-b9d0-00fd2203eb1b service placement] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:47.646659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71591aa-f74a-4f29-b9d0-00fd2203eb1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:47.661121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71591aa-f74a-4f29-b9d0-00fd2203eb1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:47.666094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71591aa-f74a-4f29-b9d0-00fd2203eb1b service placement] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:47.681852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 890/1769] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:47.683355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ecde4-a73f-4f7e-82ca-4444561458e8 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:47.683654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ecde4-a73f-4f7e-82ca-4444561458e8 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:47.683935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ecde4-a73f-4f7e-82ca-4444561458e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:47.724127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-9a4ecde4-a73f-4f7e-82ca-4444561458e8 None None] MFA Rules not processed for user `850e86e60d704092a1bb5a96db587196`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:47.725245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9a4ecde4-a73f-4f7e-82ca-4444561458e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:47.849017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 891/1770] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:47.858482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ecde4-a73f-4f7e-82ca-4444561458e8 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:47.870230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190138c-4acf-4ddf-bd85-821d4c6f94a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:47.870968 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190138c-4acf-4ddf-bd85-821d4c6f94a9 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:47.871497 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190138c-4acf-4ddf-bd85-821d4c6f94a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:47.871675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190138c-4acf-4ddf-bd85-821d4c6f94a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:47.888539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:47.893650 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:47.981903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ecde4-a73f-4f7e-82ca-4444561458e8 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:47.990223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96fccc-dd04-4256-b89b-c6489199ab82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:47.991789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96fccc-dd04-4256-b89b-c6489199ab82 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:47.992414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96fccc-dd04-4256-b89b-c6489199ab82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:47.992952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96fccc-dd04-4256-b89b-c6489199ab82 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:47.994072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96fccc-dd04-4256-b89b-c6489199ab82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:47.997125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96fccc-dd04-4256-b89b-c6489199ab82 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:48.043860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 892/1771] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:47 2022] POST /identity/v3/projects => generated 353 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:48.048779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96fccc-dd04-4256-b89b-c6489199ab82 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:48.077454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190138c-4acf-4ddf-bd85-821d4c6f94a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:48.093556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190138c-4acf-4ddf-bd85-821d4c6f94a9 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:48.160029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 880/1772] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:47 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:48.279047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96fccc-dd04-4256-b89b-c6489199ab82 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:48.303121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86981f0-ce7f-4cbc-90ad-e567089f7275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:48.304361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86981f0-ce7f-4cbc-90ad-e567089f7275 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:48.304879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86981f0-ce7f-4cbc-90ad-e567089f7275 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:48.309565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86981f0-ce7f-4cbc-90ad-e567089f7275 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:48.311303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86981f0-ce7f-4cbc-90ad-e567089f7275 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:48.323961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86981f0-ce7f-4cbc-90ad-e567089f7275 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:48.416190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 893/1773] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:48 2022] POST /identity/v3/users => generated 283 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:48.422876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190138c-4acf-4ddf-bd85-821d4c6f94a9 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:48.442735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190138c-4acf-4ddf-bd85-821d4c6f94a9 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:48.447740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86981f0-ce7f-4cbc-90ad-e567089f7275 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:48.475354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1d4cb-7f7a-4dfd-a559-3a8e5d5bc86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:48.479285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1d4cb-7f7a-4dfd-a559-3a8e5d5bc86f admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:48.479839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1d4cb-7f7a-4dfd-a559-3a8e5d5bc86f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:48.482168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1d4cb-7f7a-4dfd-a559-3a8e5d5bc86f admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:48.485929 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1d4cb-7f7a-4dfd-a559-3a8e5d5bc86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:48.495724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1d4cb-7f7a-4dfd-a559-3a8e5d5bc86f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:48.548448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 881/1774] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:48.555430 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1d4cb-7f7a-4dfd-a559-3a8e5d5bc86f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:48.593725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1d4cb-7f7a-4dfd-a559-3a8e5d5bc86f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:48.608806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01207223-94ee-4a7c-b682-19a8c79e8c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:48.609776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01207223-94ee-4a7c-b682-19a8c79e8c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:48.610065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01207223-94ee-4a7c-b682-19a8c79e8c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:48.610923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01207223-94ee-4a7c-b682-19a8c79e8c6e admin admin] PATH_INFO: `/v3/projects/18e03663608447d0a0e4e9d4ec244b60/users/7552108b01a84a379f96931f6585c5ad/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:48.640465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01207223-94ee-4a7c-b682-19a8c79e8c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e03663608447d0a0e4e9d4ec244b60, user_id=7552108b01a84a379f96931f6585c5ad, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:48.647401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01207223-94ee-4a7c-b682-19a8c79e8c6e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:48.686573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 882/1775] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:48 2022] PUT /identity/v3/projects/18e03663608447d0a0e4e9d4ec244b60/users/7552108b01a84a379f96931f6585c5ad/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:48.693640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b51c73-9ca2-4710-9a26-244b111475e2 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:48.693843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b51c73-9ca2-4710-9a26-244b111475e2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:48.695265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b51c73-9ca2-4710-9a26-244b111475e2 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:48.704649 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 883/1776] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 20:54:48 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:54:48.707817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b51c73-9ca2-4710-9a26-244b111475e2 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:48.737459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86981f0-ce7f-4cbc-90ad-e567089f7275 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:48.912234 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:48.933615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b51c73-9ca2-4710-9a26-244b111475e2 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:48.941218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2417b1-b19a-44a3-a22b-034d7aafa919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:48.941926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2417b1-b19a-44a3-a22b-034d7aafa919 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:48.942261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070534f-2977-481d-8982-0c4ab56e8011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:48.942379 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2417b1-b19a-44a3-a22b-034d7aafa919 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:48.942605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2417b1-b19a-44a3-a22b-034d7aafa919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:48.943423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070534f-2977-481d-8982-0c4ab56e8011 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:48.943889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070534f-2977-481d-8982-0c4ab56e8011 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:48.944384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070534f-2977-481d-8982-0c4ab56e8011 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:48.947446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7b2417b1-b19a-44a3-a22b-034d7aafa919 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:48.948646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070534f-2977-481d-8982-0c4ab56e8011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:48.952487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070534f-2977-481d-8982-0c4ab56e8011 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:48.974540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 884/1777] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:48.974540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36060883-5d13-4b52-a73d-0295ba4c0d5d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:48.974540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36060883-5d13-4b52-a73d-0295ba4c0d5d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:48.974540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36060883-5d13-4b52-a73d-0295ba4c0d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:49.004526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-36060883-5d13-4b52-a73d-0295ba4c0d5d None None] MFA Rules not processed for user `fffebfc51dba47f68263c0bcd6b18d22`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:49.006085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-36060883-5d13-4b52-a73d-0295ba4c0d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:49.168019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:49.194972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 885/1778] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 20:54:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:49.196589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2417b1-b19a-44a3-a22b-034d7aafa919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:49.197690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36060883-5d13-4b52-a73d-0295ba4c0d5d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:49.199983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2417b1-b19a-44a3-a22b-034d7aafa919 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:49.227839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 894/1779] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 781 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:49.230771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2417b1-b19a-44a3-a22b-034d7aafa919 service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:49.261040 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36060883-5d13-4b52-a73d-0295ba4c0d5d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:49.269037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2417b1-b19a-44a3-a22b-034d7aafa919 service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:49.286551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be49b61-ea13-406f-a081-b4efd569b989 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:49.287244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be49b61-ea13-406f-a081-b4efd569b989 service placement] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:49.288394 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be49b61-ea13-406f-a081-b4efd569b989 service placement] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:49.288394 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be49b61-ea13-406f-a081-b4efd569b989 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:49.290015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501334a-bdde-4a08-b106-cd51471cfe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:49.290992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501334a-bdde-4a08-b106-cd51471cfe97 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:49.295056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501334a-bdde-4a08-b106-cd51471cfe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:49.295537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501334a-bdde-4a08-b106-cd51471cfe97 admin admin] PATH_INFO: `/v3/projects/18e03663608447d0a0e4e9d4ec244b60/users/7552108b01a84a379f96931f6585c5ad/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:49.300167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be49b61-ea13-406f-a081-b4efd569b989 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:49.303081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be49b61-ea13-406f-a081-b4efd569b989 service placement] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:49.306689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501334a-bdde-4a08-b106-cd51471cfe97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e03663608447d0a0e4e9d4ec244b60, user_id=7552108b01a84a379f96931f6585c5ad, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:49.321648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 895/1780] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:49.323350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501334a-bdde-4a08-b106-cd51471cfe97 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:49.386944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be49b61-ea13-406f-a081-b4efd569b989 service placement] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:49.389626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3be49b61-ea13-406f-a081-b4efd569b989 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:49.423968 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 886/1781] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:49 2022] PUT /identity/v3/projects/18e03663608447d0a0e4e9d4ec244b60/users/7552108b01a84a379f96931f6585c5ad/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:49.427823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be49b61-ea13-406f-a081-b4efd569b989 service placement] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:49.435133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ebf71-12e7-4b56-9151-78f6035a7502 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:49.435605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ebf71-12e7-4b56-9151-78f6035a7502 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:49.436193 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ebf71-12e7-4b56-9151-78f6035a7502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:49.568304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-824ebf71-12e7-4b56-9151-78f6035a7502 None None] MFA Rules not processed for user `7552108b01a84a379f96931f6585c5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:49.574964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-824ebf71-12e7-4b56-9151-78f6035a7502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:49.584123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:49.585239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:49.585640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:49.585985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] PATH_INFO: `/v3/projects/242ba4b38fd24d388fbddda34faf1881` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:49.587431 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=242ba4b38fd24d388fbddda34faf1881)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:49.590260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:49.793319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242ba4b38fd24d388fbddda34faf1881', 'request_id': 'req-a0737cc2-f851-4f19-9675-fdb64fb5c842'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:49.803217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 887/1782] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:54:49.816934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242ba4b38fd24d388fbddda34faf1881', 'request_id': 'req-a0737cc2-f851-4f19-9675-fdb64fb5c842'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:49.833925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242ba4b38fd24d388fbddda34faf1881', 'request_id': 'req-a0737cc2-f851-4f19-9675-fdb64fb5c842'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:49.865086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] The token cache is being invalidate because project 242ba4b38fd24d388fbddda34faf1881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:49.865434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:49.868288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:49.871612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 896/1783] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:49 2022] DELETE /identity/v3/projects/242ba4b38fd24d388fbddda34faf1881 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:49.877661 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ec607-4c94-4d39-852e-57aa31524774 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:49.878024 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ec607-4c94-4d39-852e-57aa31524774 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:49.878433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ec607-4c94-4d39-852e-57aa31524774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:49.915697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:49.918309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:49.973685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-784ec607-4c94-4d39-852e-57aa31524774 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:49.975668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-784ec607-4c94-4d39-852e-57aa31524774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:50.049350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:50.214145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 888/1784] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:54:50.220532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caed56-d6d7-4572-9f2a-5d163f97dbfc None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:50.220818 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caed56-d6d7-4572-9f2a-5d163f97dbfc None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:50.221102 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caed56-d6d7-4572-9f2a-5d163f97dbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:50.246046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-63caed56-d6d7-4572-9f2a-5d163f97dbfc None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:50.248412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-63caed56-d6d7-4572-9f2a-5d163f97dbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:50.264488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 889/1785] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:50.265803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caed56-d6d7-4572-9f2a-5d163f97dbfc None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:50.277659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caed56-d6d7-4572-9f2a-5d163f97dbfc None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:50.293541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0ea92-db65-4e6d-bd72-ff1d50b5437b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:50.297837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0ea92-db65-4e6d-bd72-ff1d50b5437b admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:50.299454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0ea92-db65-4e6d-bd72-ff1d50b5437b admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:50.300151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0ea92-db65-4e6d-bd72-ff1d50b5437b admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:50.302530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0ea92-db65-4e6d-bd72-ff1d50b5437b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:50.302880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0ea92-db65-4e6d-bd72-ff1d50b5437b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:50.310908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0ea92-db65-4e6d-bd72-ff1d50b5437b admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:50.322709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0737cc2-f851-4f19-9675-fdb64fb5c842 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:50.352827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1ad3d-25a5-447d-bf8d-325fa43b76b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:50.357197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 890/1786] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:50.379437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0ea92-db65-4e6d-bd72-ff1d50b5437b admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:50.379998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ad3d-25a5-447d-bf8d-325fa43b76b1 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:50.379998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ad3d-25a5-447d-bf8d-325fa43b76b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:50.379998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ad3d-25a5-447d-bf8d-325fa43b76b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:50.381809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7fb1ad3d-25a5-447d-bf8d-325fa43b76b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:50.388716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0ea92-db65-4e6d-bd72-ff1d50b5437b admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:50.396492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6349f-d135-454f-851b-b2768832c764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:50.397342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6349f-d135-454f-851b-b2768832c764 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:50.397786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6349f-d135-454f-851b-b2768832c764 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:50.398119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6349f-d135-454f-851b-b2768832c764 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:50.399381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6349f-d135-454f-851b-b2768832c764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:50.401479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6349f-d135-454f-851b-b2768832c764 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:50.467206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 891/1787] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:50 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:50.470820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6349f-d135-454f-851b-b2768832c764 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:50.577241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:50.726748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1ad3d-25a5-447d-bf8d-325fa43b76b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:50.735474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1ad3d-25a5-447d-bf8d-325fa43b76b1 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:50.802320 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6349f-d135-454f-851b-b2768832c764 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:50.810461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 897/1788] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:49 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 896 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:50.817282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c98e5-f58b-4e8a-a13c-330e62e6d41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:50.818800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c98e5-f58b-4e8a-a13c-330e62e6d41c admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:50.819271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c98e5-f58b-4e8a-a13c-330e62e6d41c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:50.819702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c98e5-f58b-4e8a-a13c-330e62e6d41c admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:50.820905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c98e5-f58b-4e8a-a13c-330e62e6d41c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:50.823110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c98e5-f58b-4e8a-a13c-330e62e6d41c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:50.899887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 892/1789] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:50 2022] POST /identity/v3/users => generated 292 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:50.904204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1ad3d-25a5-447d-bf8d-325fa43b76b1 service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:50.924899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1ad3d-25a5-447d-bf8d-325fa43b76b1 service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:50.941209 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7f25e-3240-4059-abc9-df2713d0d412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:50.942301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7f25e-3240-4059-abc9-df2713d0d412 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:50.942595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7f25e-3240-4059-abc9-df2713d0d412 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:50.942877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7f25e-3240-4059-abc9-df2713d0d412 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:50.944908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7f25e-3240-4059-abc9-df2713d0d412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:50.949964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7f25e-3240-4059-abc9-df2713d0d412 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:50.975048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 898/1790] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:50.981552 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c98e5-f58b-4e8a-a13c-330e62e6d41c admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:51.002189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c98e5-f58b-4e8a-a13c-330e62e6d41c admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:51.016638 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693678d-4d75-4687-b261-86bb6fd00035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:51.019683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7f25e-3240-4059-abc9-df2713d0d412 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:51.034192 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693678d-4d75-4687-b261-86bb6fd00035 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:51.034868 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693678d-4d75-4687-b261-86bb6fd00035 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:51.036883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693678d-4d75-4687-b261-86bb6fd00035 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:51.041841 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693678d-4d75-4687-b261-86bb6fd00035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:51.049722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693678d-4d75-4687-b261-86bb6fd00035 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:51.113001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 893/1791] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:51.113839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693678d-4d75-4687-b261-86bb6fd00035 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:51.137276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693678d-4d75-4687-b261-86bb6fd00035 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:51.167490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ccf06-6002-406f-b248-c29c6f31be85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:51.170514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ccf06-6002-406f-b248-c29c6f31be85 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:51.170840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ccf06-6002-406f-b248-c29c6f31be85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:51.172444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ccf06-6002-406f-b248-c29c6f31be85 admin admin] PATH_INFO: `/v3/projects/52d379671ea64a5fbfe4817dfac4006c/users/7a70545a52004b869bbc0aee0cd83950/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:51.246214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7f25e-3240-4059-abc9-df2713d0d412 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:51.250304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fdcc42-e050-4a34-a6b7-93c333d5cc47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:51.251107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdcc42-e050-4a34-a6b7-93c333d5cc47 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:51.251482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdcc42-e050-4a34-a6b7-93c333d5cc47 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:51.251825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fdcc42-e050-4a34-a6b7-93c333d5cc47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:51.254060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-67fdcc42-e050-4a34-a6b7-93c333d5cc47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:51.264571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ccf06-6002-406f-b248-c29c6f31be85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d379671ea64a5fbfe4817dfac4006c, user_id=7a70545a52004b869bbc0aee0cd83950, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:51.271676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ccf06-6002-406f-b248-c29c6f31be85 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:51.324797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 894/1792] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:51 2022] PUT /identity/v3/projects/52d379671ea64a5fbfe4817dfac4006c/users/7a70545a52004b869bbc0aee0cd83950/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:51.326070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ccf06-6002-406f-b248-c29c6f31be85 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:51.329489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-806ccf06-6002-406f-b248-c29c6f31be85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:51.376188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fdcc42-e050-4a34-a6b7-93c333d5cc47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:51.387838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fdcc42-e050-4a34-a6b7-93c333d5cc47 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:51.410269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 899/1793] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:51 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:54:51.413185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d12cd9-e458-4b38-b2be-5dfddb7053b6 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:51.413670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d12cd9-e458-4b38-b2be-5dfddb7053b6 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:51.414197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d12cd9-e458-4b38-b2be-5dfddb7053b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:51.457163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:51.505672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ccf06-6002-406f-b248-c29c6f31be85 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:51.505919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-f2d12cd9-e458-4b38-b2be-5dfddb7053b6 None None] MFA Rules not processed for user `7a70545a52004b869bbc0aee0cd83950`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:51.508401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f2d12cd9-e458-4b38-b2be-5dfddb7053b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:51.517457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:51.519554 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:51.521114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:51.522965 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] PATH_INFO: `/v3/users/8a14450faa014e1ebae3fe7f8e07d936` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:51.525445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a14450faa014e1ebae3fe7f8e07d936)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:51.530809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:51.691641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 900/1794] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:51.701927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d12cd9-e458-4b38-b2be-5dfddb7053b6 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:51.702739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a14450faa014e1ebae3fe7f8e07d936'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:51.723672 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d12cd9-e458-4b38-b2be-5dfddb7053b6 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:51.731604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a14450faa014e1ebae3fe7f8e07d936'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:51.733574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6eb8d7-24df-4f28-ad8c-e39bb3b713d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:51.738755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6eb8d7-24df-4f28-ad8c-e39bb3b713d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:51.738755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6eb8d7-24df-4f28-ad8c-e39bb3b713d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:51.738755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6eb8d7-24df-4f28-ad8c-e39bb3b713d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:51.738755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6eb8d7-24df-4f28-ad8c-e39bb3b713d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:51.744016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6eb8d7-24df-4f28-ad8c-e39bb3b713d8 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:51.774713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a14450faa014e1ebae3fe7f8e07d936'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:51.778328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a14450faa014e1ebae3fe7f8e07d936'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:51.800247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a14450faa014e1ebae3fe7f8e07d936'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:51.810610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 895/1795] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:51 2022] DELETE /identity/v3/users/8a14450faa014e1ebae3fe7f8e07d936 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:51.819383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 901/1796] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:51 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:51.824791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:51.826548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:51.851373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:52.204975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469e50a-b4d2-42cf-9383-e6a490d67b9b admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:52.425500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93590ec0-3a12-4d14-b909-45956cc505a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:52.428607 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93590ec0-3a12-4d14-b909-45956cc505a6 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:52.429433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93590ec0-3a12-4d14-b909-45956cc505a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:52.430541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93590ec0-3a12-4d14-b909-45956cc505a6 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:52.435741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93590ec0-3a12-4d14-b909-45956cc505a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:52.457433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93590ec0-3a12-4d14-b909-45956cc505a6 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:52.528548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 896/1797] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:51 2022] POST /identity/v3/users => generated 293 bytes in 699 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:54:52.529788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6eb8d7-24df-4f28-ad8c-e39bb3b713d8 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:52.555596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6eb8d7-24df-4f28-ad8c-e39bb3b713d8 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:52.556901 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93590ec0-3a12-4d14-b909-45956cc505a6 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:52.561733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-93590ec0-3a12-4d14-b909-45956cc505a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:52.567762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c213d-4b5d-489b-8075-a4d00f033ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:52.569255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c213d-4b5d-489b-8075-a4d00f033ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:52.569770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c213d-4b5d-489b-8075-a4d00f033ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:52.570574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c213d-4b5d-489b-8075-a4d00f033ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:52.572148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c213d-4b5d-489b-8075-a4d00f033ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:52.576105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c213d-4b5d-489b-8075-a4d00f033ac8 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:52.601223 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93590ec0-3a12-4d14-b909-45956cc505a6 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:52.613611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:52.614848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:52.615478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:52.616126 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] PATH_INFO: `/v3/users/e62a6f741eee42f4a9138c6b26cf3d29` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:52.619246 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=e62a6f741eee42f4a9138c6b26cf3d29)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:52.624371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 902/1798] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:54:52.624636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:52.632270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c213d-4b5d-489b-8075-a4d00f033ac8 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:52.679924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c213d-4b5d-489b-8075-a4d00f033ac8 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:52.700385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebeda6-e8cb-4773-b534-31f11a72028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:52.702388 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebeda6-e8cb-4773-b534-31f11a72028f admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:52.703015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebeda6-e8cb-4773-b534-31f11a72028f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:52.704807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebeda6-e8cb-4773-b534-31f11a72028f admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:52.706905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebeda6-e8cb-4773-b534-31f11a72028f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:52.710964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebeda6-e8cb-4773-b534-31f11a72028f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:52.724559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e62a6f741eee42f4a9138c6b26cf3d29'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:52.746415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 903/1799] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:52.746415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebeda6-e8cb-4773-b534-31f11a72028f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:52.752212 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a6ebeda6-e8cb-4773-b534-31f11a72028f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:52.759502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e62a6f741eee42f4a9138c6b26cf3d29'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:52.770969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e62a6f741eee42f4a9138c6b26cf3d29'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:52.772145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e62a6f741eee42f4a9138c6b26cf3d29'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:52.823334 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e62a6f741eee42f4a9138c6b26cf3d29'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:52.840599 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 897/1800] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:52 2022] DELETE /identity/v3/users/e62a6f741eee42f4a9138c6b26cf3d29 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:52.840599 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:52.844050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebeda6-e8cb-4773-b534-31f11a72028f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:52.848891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:52.896137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:53.014290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:53.016022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:53.016902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:53.017690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] PATH_INFO: `/v3/projects/522264527e0e4130b615936b03e90e68` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:53.020373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=522264527e0e4130b615936b03e90e68)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:53.024354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:53.169646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '522264527e0e4130b615936b03e90e68', 'request_id': 'req-20958d2f-a1df-4517-9ca8-974096fee2bb'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:53.195771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '522264527e0e4130b615936b03e90e68', 'request_id': 'req-20958d2f-a1df-4517-9ca8-974096fee2bb'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:53.198867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:53.225903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d682270-f817-4285-9594-c43fbe1436ff admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:53.228368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '522264527e0e4130b615936b03e90e68', 'request_id': 'req-20958d2f-a1df-4517-9ca8-974096fee2bb'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:53.289300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] The token cache is being invalidate because project 522264527e0e4130b615936b03e90e68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:53.289940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:53.295254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20958d2f-a1df-4517-9ca8-974096fee2bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:53.301779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 904/1801] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:52 2022] DELETE /identity/v3/projects/522264527e0e4130b615936b03e90e68 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:53.307177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454009ee-5dc3-48ff-88ce-05ffd27f60cd None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:53.308464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454009ee-5dc3-48ff-88ce-05ffd27f60cd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:53.308897 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454009ee-5dc3-48ff-88ce-05ffd27f60cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:53.387701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-454009ee-5dc3-48ff-88ce-05ffd27f60cd None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:53.394502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-454009ee-5dc3-48ff-88ce-05ffd27f60cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:53.452063 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ecf048-7215-4c78-ae8f-46efb9c1d7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:53.453047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecf048-7215-4c78-ae8f-46efb9c1d7ce admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:53.453316 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecf048-7215-4c78-ae8f-46efb9c1d7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:53.453594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecf048-7215-4c78-ae8f-46efb9c1d7ce admin admin] PATH_INFO: `/v3/projects/50efcf197176480989d8b3935799b1eb/users/da262dff9ad846dc81ae641256fef6ab/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:53.487483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ecf048-7215-4c78-ae8f-46efb9c1d7ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=50efcf197176480989d8b3935799b1eb, user_id=da262dff9ad846dc81ae641256fef6ab, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:53.490262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ecf048-7215-4c78-ae8f-46efb9c1d7ce admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:53.514928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 898/1802] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:52 2022] PUT /identity/v3/projects/50efcf197176480989d8b3935799b1eb/users/da262dff9ad846dc81ae641256fef6ab/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:53.515769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ecf048-7215-4c78-ae8f-46efb9c1d7ce admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:53.526664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b1ecf048-7215-4c78-ae8f-46efb9c1d7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:53.540314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 905/1803] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:53.542243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6c26a-2323-4a3f-8f87-1aced14acdce None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:53.542706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6c26a-2323-4a3f-8f87-1aced14acdce None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:53.543187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6c26a-2323-4a3f-8f87-1aced14acdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:53.650133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-d8d6c26a-2323-4a3f-8f87-1aced14acdce None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:53.653045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d8d6c26a-2323-4a3f-8f87-1aced14acdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:53.689240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ecf048-7215-4c78-ae8f-46efb9c1d7ce admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:53.700707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:53.702991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:53.704766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:53.705167 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] PATH_INFO: `/v3/users/419f34f913c449afab926ef39471805a` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:53.707262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=419f34f913c449afab926ef39471805a)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:53.714227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:53.824937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:53.833068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '419f34f913c449afab926ef39471805a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:53.845333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '419f34f913c449afab926ef39471805a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:53.859852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '419f34f913c449afab926ef39471805a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:53.861129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '419f34f913c449afab926ef39471805a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:53.868089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 906/1804] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:53.870790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775915-ea1e-45e2-8fba-b15f20dee553 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:53.871328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775915-ea1e-45e2-8fba-b15f20dee553 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:53.872173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22775915-ea1e-45e2-8fba-b15f20dee553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:53.881965 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '419f34f913c449afab926ef39471805a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:53.892921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 899/1805] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:53 2022] DELETE /identity/v3/users/419f34f913c449afab926ef39471805a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:53.894326 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:53.896019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:53.928454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-22775915-ea1e-45e2-8fba-b15f20dee553 None None] MFA Rules not processed for user `da262dff9ad846dc81ae641256fef6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:53.929829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-22775915-ea1e-45e2-8fba-b15f20dee553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:53.993848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:54.004361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0363a40-be11-4a63-8222-50dca64d1733 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:54.011911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701662d-85bd-45d4-bdfc-0685389f2e15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:54.012884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701662d-85bd-45d4-bdfc-0685389f2e15 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:54.013288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701662d-85bd-45d4-bdfc-0685389f2e15 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:54.013661 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701662d-85bd-45d4-bdfc-0685389f2e15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:54.015738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8701662d-85bd-45d4-bdfc-0685389f2e15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:54.042839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 907/1806] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:54.046833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22775915-ea1e-45e2-8fba-b15f20dee553 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:54.050212 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-22775915-ea1e-45e2-8fba-b15f20dee553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:54.114458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701662d-85bd-45d4-bdfc-0685389f2e15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:54.116304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701662d-85bd-45d4-bdfc-0685389f2e15 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:54.133597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 900/1807] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:54.135663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701662d-85bd-45d4-bdfc-0685389f2e15 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:54.137464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8701662d-85bd-45d4-bdfc-0685389f2e15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:54.170215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22775915-ea1e-45e2-8fba-b15f20dee553 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:54.183599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d504ace-a54a-4a42-a993-c38c3c6488a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:54.184311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d504ace-a54a-4a42-a993-c38c3c6488a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:54.184622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d504ace-a54a-4a42-a993-c38c3c6488a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:54.184904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d504ace-a54a-4a42-a993-c38c3c6488a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:54.185619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d504ace-a54a-4a42-a993-c38c3c6488a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:54.187700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d504ace-a54a-4a42-a993-c38c3c6488a5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:54.251500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 908/1808] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:54 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:54.255136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d504ace-a54a-4a42-a993-c38c3c6488a5 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:54.326042 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:54.395899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d504ace-a54a-4a42-a993-c38c3c6488a5 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:54.401779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b1e88-87ac-4f8f-9a86-82f844842983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:54.402629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b1e88-87ac-4f8f-9a86-82f844842983 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:54.402919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b1e88-87ac-4f8f-9a86-82f844842983 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:54.403203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b1e88-87ac-4f8f-9a86-82f844842983 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:54.403913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b1e88-87ac-4f8f-9a86-82f844842983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:54.406389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b1e88-87ac-4f8f-9a86-82f844842983 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:54.409456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701662d-85bd-45d4-bdfc-0685389f2e15 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:54.441328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 909/1809] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:54 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:54:54.457423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b1e88-87ac-4f8f-9a86-82f844842983 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:54.475569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b1e88-87ac-4f8f-9a86-82f844842983 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:54.485777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7d19f-56d1-4c7f-9525-77d2c7b0491e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:54.486469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7d19f-56d1-4c7f-9525-77d2c7b0491e admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:54.486752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7d19f-56d1-4c7f-9525-77d2c7b0491e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:54.487041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7d19f-56d1-4c7f-9525-77d2c7b0491e admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:54.487763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7d19f-56d1-4c7f-9525-77d2c7b0491e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:54.489562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f7d19f-56d1-4c7f-9525-77d2c7b0491e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:54.512613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe72d1-5283-469c-8199-a482cbe33991 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:54.513722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 910/1810] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:54.514693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7d19f-56d1-4c7f-9525-77d2c7b0491e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:54.515865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe72d1-5283-469c-8199-a482cbe33991 service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:54.516178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe72d1-5283-469c-8199-a482cbe33991 service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:54.517936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe72d1-5283-469c-8199-a482cbe33991 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:54.522562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bffe72d1-5283-469c-8199-a482cbe33991 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:54.541616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7d19f-56d1-4c7f-9525-77d2c7b0491e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:54.557708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83d529-a1f2-4ceb-8f0c-6573a4522688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:54.558620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83d529-a1f2-4ceb-8f0c-6573a4522688 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:54.558905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83d529-a1f2-4ceb-8f0c-6573a4522688 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:54.559184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83d529-a1f2-4ceb-8f0c-6573a4522688 admin admin] PATH_INFO: `/v3/projects/d1c3c0ffb2184938a1a7f38897173305/users/b70d47088d2e4208a89696a7c970d9e9/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:54.602401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83d529-a1f2-4ceb-8f0c-6573a4522688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c3c0ffb2184938a1a7f38897173305, user_id=b70d47088d2e4208a89696a7c970d9e9, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:54.605205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83d529-a1f2-4ceb-8f0c-6573a4522688 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:54.632068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe72d1-5283-469c-8199-a482cbe33991 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:54.636989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 911/1811] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:54 2022] PUT /identity/v3/projects/d1c3c0ffb2184938a1a7f38897173305/users/b70d47088d2e4208a89696a7c970d9e9/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:54.637700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe72d1-5283-469c-8199-a482cbe33991 service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:54.648422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83d529-a1f2-4ceb-8f0c-6573a4522688 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:54.675652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 901/1812] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 20:54:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:54:54.745712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:54.810548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe72d1-5283-469c-8199-a482cbe33991 service swift] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:54.813040 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bffe72d1-5283-469c-8199-a482cbe33991 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:54.835044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe72d1-5283-469c-8199-a482cbe33991 service swift] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:54.921910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83d529-a1f2-4ceb-8f0c-6573a4522688 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:54.933647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ff31e-18d5-4891-a714-9eea7d417f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:54.934807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ff31e-18d5-4891-a714-9eea7d417f61 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:54.935197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ff31e-18d5-4891-a714-9eea7d417f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:54.935579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ff31e-18d5-4891-a714-9eea7d417f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:54.936982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ff31e-18d5-4891-a714-9eea7d417f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:54.940936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ff31e-18d5-4891-a714-9eea7d417f61 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:54.980013 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 912/1813] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:54.991396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ff31e-18d5-4891-a714-9eea7d417f61 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:55.042241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:55.043184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ff31e-18d5-4891-a714-9eea7d417f61 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:55.046567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:55.049866 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:55.051391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] PATH_INFO: `/v3/projects/ede33262046e4f2e937fd47701417164` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:55.059585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ede33262046e4f2e937fd47701417164)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:55.065509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8c26e-66fc-45aa-b230-7f35a57139c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:55.068673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:55.070731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c26e-66fc-45aa-b230-7f35a57139c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:55.073389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c26e-66fc-45aa-b230-7f35a57139c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:55.074325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c26e-66fc-45aa-b230-7f35a57139c0 admin admin] PATH_INFO: `/v3/projects/d1c3c0ffb2184938a1a7f38897173305/users/b70d47088d2e4208a89696a7c970d9e9/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:55.082503 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8c26e-66fc-45aa-b230-7f35a57139c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c3c0ffb2184938a1a7f38897173305, user_id=b70d47088d2e4208a89696a7c970d9e9, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:55.087588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8c26e-66fc-45aa-b230-7f35a57139c0 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:55.126054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 913/1814] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:54 2022] PUT /identity/v3/projects/d1c3c0ffb2184938a1a7f38897173305/users/b70d47088d2e4208a89696a7c970d9e9/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:55.135553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5b0a-ebb7-4db9-90ee-6c3889754b59 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:55.136826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5b0a-ebb7-4db9-90ee-6c3889754b59 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:55.138055 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5b0a-ebb7-4db9-90ee-6c3889754b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:55.229411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-9fdc5b0a-ebb7-4db9-90ee-6c3889754b59 None None] MFA Rules not processed for user `b70d47088d2e4208a89696a7c970d9e9`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:55.231685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9fdc5b0a-ebb7-4db9-90ee-6c3889754b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:55.267904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:55.317374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ede33262046e4f2e937fd47701417164', 'request_id': 'req-8f1a09ff-c506-4879-b354-046ed4adbb19'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:55.346401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ede33262046e4f2e937fd47701417164', 'request_id': 'req-8f1a09ff-c506-4879-b354-046ed4adbb19'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:55.411707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ede33262046e4f2e937fd47701417164', 'request_id': 'req-8f1a09ff-c506-4879-b354-046ed4adbb19'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:55.455591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] The token cache is being invalidate because project ede33262046e4f2e937fd47701417164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:55.455591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:55.464314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-8f1a09ff-c506-4879-b354-046ed4adbb19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:55.472213 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 902/1815] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:54 2022] DELETE /identity/v3/projects/ede33262046e4f2e937fd47701417164 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:55.479123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327c3bf-9a0f-4f02-bd81-d347c0a2e57c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:55.479328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327c3bf-9a0f-4f02-bd81-d347c0a2e57c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:55.479513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327c3bf-9a0f-4f02-bd81-d347c0a2e57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:55.533416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-8327c3bf-9a0f-4f02-bd81-d347c0a2e57c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:55.535766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8327c3bf-9a0f-4f02-bd81-d347c0a2e57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:55.542524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 914/1816] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:54:55.555716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc5b0a-ebb7-4db9-90ee-6c3889754b59 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:55.566851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9fdc5b0a-ebb7-4db9-90ee-6c3889754b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:55.796037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 903/1817] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:55.806753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2322b-12d4-4877-9fc3-c6945615bcea None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:55.807188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2322b-12d4-4877-9fc3-c6945615bcea None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:55.807692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2322b-12d4-4877-9fc3-c6945615bcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:55.867893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc5b0a-ebb7-4db9-90ee-6c3889754b59 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:55.898300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-bee2322b-12d4-4877-9fc3-c6945615bcea None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:55.900799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bee2322b-12d4-4877-9fc3-c6945615bcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:55.977805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 904/1818] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:55.977805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2322b-12d4-4877-9fc3-c6945615bcea None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:56.025618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2322b-12d4-4877-9fc3-c6945615bcea None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:56.045446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61870545-08d5-435c-a005-c9f0100c91aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:56.046666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61870545-08d5-435c-a005-c9f0100c91aa service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:56.047060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61870545-08d5-435c-a005-c9f0100c91aa service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:56.047447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61870545-08d5-435c-a005-c9f0100c91aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:56.053947 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-61870545-08d5-435c-a005-c9f0100c91aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:56.058151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04dc626-55f7-4db0-a6df-99464c2a04c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:56.059284 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dc626-55f7-4db0-a6df-99464c2a04c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:56.059780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dc626-55f7-4db0-a6df-99464c2a04c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:56.060281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04dc626-55f7-4db0-a6df-99464c2a04c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:56.061693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04dc626-55f7-4db0-a6df-99464c2a04c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:56.062243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04dc626-55f7-4db0-a6df-99464c2a04c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:56.066148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04dc626-55f7-4db0-a6df-99464c2a04c1 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:56.067456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:56.113264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 905/1819] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:56.117864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04dc626-55f7-4db0-a6df-99464c2a04c1 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:56.137169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04dc626-55f7-4db0-a6df-99464c2a04c1 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:56.147331 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a1999-1f6c-4d17-8343-2a71eac09edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:56.149007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a1999-1f6c-4d17-8343-2a71eac09edf admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:56.149617 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a1999-1f6c-4d17-8343-2a71eac09edf admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:56.150119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a1999-1f6c-4d17-8343-2a71eac09edf admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:56.151665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a1999-1f6c-4d17-8343-2a71eac09edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:56.154229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a1999-1f6c-4d17-8343-2a71eac09edf admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:56.186126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61870545-08d5-435c-a005-c9f0100c91aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:56.187648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61870545-08d5-435c-a005-c9f0100c91aa service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:56.198812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 915/1821] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:56.199091 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 906/1821] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:56 2022] POST /identity/v3/projects => generated 365 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:54:56.199795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61870545-08d5-435c-a005-c9f0100c91aa service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:56.213816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61870545-08d5-435c-a005-c9f0100c91aa service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:56.304588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71911c05-e860-4bdb-9b68-df1360bd529b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:56.305519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71911c05-e860-4bdb-9b68-df1360bd529b admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:56.305912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71911c05-e860-4bdb-9b68-df1360bd529b admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:56.306699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71911c05-e860-4bdb-9b68-df1360bd529b admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:56.307827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71911c05-e860-4bdb-9b68-df1360bd529b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:56.310844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71911c05-e860-4bdb-9b68-df1360bd529b admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:56.346071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 916/1822] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:56 2022] POST /identity/v3/users => generated 283 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:56.352464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a1999-1f6c-4d17-8343-2a71eac09edf admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:56.366789 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a1999-1f6c-4d17-8343-2a71eac09edf admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:56.373536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ccf8c-8389-4568-9c17-38ef2a0c6c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:56.374653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ccf8c-8389-4568-9c17-38ef2a0c6c36 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:56.375394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ccf8c-8389-4568-9c17-38ef2a0c6c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:56.375729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ccf8c-8389-4568-9c17-38ef2a0c6c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:56.376473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ccf8c-8389-4568-9c17-38ef2a0c6c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:56.378123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ccf8c-8389-4568-9c17-38ef2a0c6c36 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:56.393100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 907/1823] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:56.400622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71911c05-e860-4bdb-9b68-df1360bd529b admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:56.415310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71911c05-e860-4bdb-9b68-df1360bd529b admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:56.430176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aef5eb-7dbd-4ced-a3b1-3906c50e68ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:56.431028 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aef5eb-7dbd-4ced-a3b1-3906c50e68ef admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:56.431430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aef5eb-7dbd-4ced-a3b1-3906c50e68ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:56.431932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aef5eb-7dbd-4ced-a3b1-3906c50e68ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:56.432929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aef5eb-7dbd-4ced-a3b1-3906c50e68ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:56.435054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aef5eb-7dbd-4ced-a3b1-3906c50e68ef admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:56.449568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 917/1824] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:56.455841 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ccf8c-8389-4568-9c17-38ef2a0c6c36 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:56.486654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ccf8c-8389-4568-9c17-38ef2a0c6c36 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:56.500936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48508d4-56d5-47ab-b0bb-71da42f5ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:56.501876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48508d4-56d5-47ab-b0bb-71da42f5ef1c admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:56.502321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48508d4-56d5-47ab-b0bb-71da42f5ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:56.502595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48508d4-56d5-47ab-b0bb-71da42f5ef1c admin admin] PATH_INFO: `/v3/projects/63853b37948843a28c082110f29be672/users/11a86fb35d244359a9e7418f93309cef/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:56.516088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aef5eb-7dbd-4ced-a3b1-3906c50e68ef admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:56.518595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c3aef5eb-7dbd-4ced-a3b1-3906c50e68ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:56.537399 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aef5eb-7dbd-4ced-a3b1-3906c50e68ef admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:56.552376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:56.552376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:56.552376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:56.552376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] PATH_INFO: `/v3/projects/f4b487eadb984d0ba7962a5dfde2aedd` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:56.553336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b487eadb984d0ba7962a5dfde2aedd)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:56.553336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:56.557264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48508d4-56d5-47ab-b0bb-71da42f5ef1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=63853b37948843a28c082110f29be672, user_id=11a86fb35d244359a9e7418f93309cef, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:56.560978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48508d4-56d5-47ab-b0bb-71da42f5ef1c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:56.581722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 908/1825] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:56 2022] PUT /identity/v3/projects/63853b37948843a28c082110f29be672/users/11a86fb35d244359a9e7418f93309cef/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:56.593464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50421bb-a3a6-4a89-849d-ea4558baea20 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:56.593874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50421bb-a3a6-4a89-849d-ea4558baea20 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:56.594304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50421bb-a3a6-4a89-849d-ea4558baea20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:56.622847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b487eadb984d0ba7962a5dfde2aedd', 'request_id': 'req-1e416e5f-9019-4776-9066-ef9277595686'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:56.629833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e50421bb-a3a6-4a89-849d-ea4558baea20 None None] MFA Rules not processed for user `11a86fb35d244359a9e7418f93309cef`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:56.631064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e50421bb-a3a6-4a89-849d-ea4558baea20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:56.634751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b487eadb984d0ba7962a5dfde2aedd', 'request_id': 'req-1e416e5f-9019-4776-9066-ef9277595686'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:56.654014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b487eadb984d0ba7962a5dfde2aedd', 'request_id': 'req-1e416e5f-9019-4776-9066-ef9277595686'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:56.678054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] The token cache is being invalidate because project f4b487eadb984d0ba7962a5dfde2aedd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:56.678735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:56.680009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:56.682286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 918/1826] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:56 2022] DELETE /identity/v3/projects/f4b487eadb984d0ba7962a5dfde2aedd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:56.687028 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:56.692339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:56.712818 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:56.714270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:56.804275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 909/1827] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:56.805430 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50421bb-a3a6-4a89-849d-ea4558baea20 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:56.808287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e50421bb-a3a6-4a89-849d-ea4558baea20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:56.827041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50421bb-a3a6-4a89-849d-ea4558baea20 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:56.827636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e416e5f-9019-4776-9066-ef9277595686 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:56.832091 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac20f9-a816-4496-86c8-4d6a87ddfbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:56.832797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac20f9-a816-4496-86c8-4d6a87ddfbd7 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:56.833085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac20f9-a816-4496-86c8-4d6a87ddfbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:56.833358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac20f9-a816-4496-86c8-4d6a87ddfbd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:56.833817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:56.834064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac20f9-a816-4496-86c8-4d6a87ddfbd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:56.835400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:56.835907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:56.836121 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac20f9-a816-4496-86c8-4d6a87ddfbd7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:56.837122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] PATH_INFO: `/v3/users/33aa784807234b42a3984dcd47e463a0` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:56.838961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33aa784807234b42a3984dcd47e463a0)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:56.843856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:56.870069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 910/1828] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:56 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:54:56.874721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac20f9-a816-4496-86c8-4d6a87ddfbd7 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:56.910965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33aa784807234b42a3984dcd47e463a0'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:56.921508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33aa784807234b42a3984dcd47e463a0'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:56.922162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33aa784807234b42a3984dcd47e463a0'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:56.937145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33aa784807234b42a3984dcd47e463a0'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:56.944847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33aa784807234b42a3984dcd47e463a0'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:56.951516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 919/1829] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:54:56 2022] DELETE /identity/v3/users/33aa784807234b42a3984dcd47e463a0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:56.959636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac20f9-a816-4496-86c8-4d6a87ddfbd7 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:57.040755 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab932f-4404-4699-a78a-fe71f3f82b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:57.041485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab932f-4404-4699-a78a-fe71f3f82b9e admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:57.041798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab932f-4404-4699-a78a-fe71f3f82b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:57.042077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab932f-4404-4699-a78a-fe71f3f82b9e admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:57.042800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab932f-4404-4699-a78a-fe71f3f82b9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:57.044866 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab932f-4404-4699-a78a-fe71f3f82b9e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:57.068269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 911/1830] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:56 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:54:57.072899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:57.075334 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:57.090633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c53a6f-cfa7-4088-96a1-4e859918f1d5 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:57.095972 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a92c48-8798-4309-b6d4-52bcb70fcdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:57.096699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a92c48-8798-4309-b6d4-52bcb70fcdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:57.097268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a92c48-8798-4309-b6d4-52bcb70fcdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:57.097268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a92c48-8798-4309-b6d4-52bcb70fcdb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:57.097995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a92c48-8798-4309-b6d4-52bcb70fcdb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:57.099702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a92c48-8798-4309-b6d4-52bcb70fcdb3 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:57.118363 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 920/1831] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:57.123687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab932f-4404-4699-a78a-fe71f3f82b9e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:57.141037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab932f-4404-4699-a78a-fe71f3f82b9e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:57.147613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12fbc5-430b-4c6d-9832-06dd4c8f0d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:57.148660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12fbc5-430b-4c6d-9832-06dd4c8f0d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:57.148943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12fbc5-430b-4c6d-9832-06dd4c8f0d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:57.149219 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12fbc5-430b-4c6d-9832-06dd4c8f0d5f admin admin] PATH_INFO: `/v3/projects/520e151d33ca450ab8a690774d6715cb/users/e20e467a63ea4d788578c880789a48b7/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:57.166194 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12fbc5-430b-4c6d-9832-06dd4c8f0d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=520e151d33ca450ab8a690774d6715cb, user_id=e20e467a63ea4d788578c880789a48b7, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:57.169175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12fbc5-430b-4c6d-9832-06dd4c8f0d5f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:57.183698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 912/1832] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:57 2022] PUT /identity/v3/projects/520e151d33ca450ab8a690774d6715cb/users/e20e467a63ea4d788578c880789a48b7/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:54:57.187845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a92c48-8798-4309-b6d4-52bcb70fcdb3 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:57.199311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a92c48-8798-4309-b6d4-52bcb70fcdb3 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:57.287128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dad1d-f45f-463c-9bf6-0c310c78ea49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:57.287985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dad1d-f45f-463c-9bf6-0c310c78ea49 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:57.288125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dad1d-f45f-463c-9bf6-0c310c78ea49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:57.288411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dad1d-f45f-463c-9bf6-0c310c78ea49 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:57.289161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dad1d-f45f-463c-9bf6-0c310c78ea49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:57.290808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dad1d-f45f-463c-9bf6-0c310c78ea49 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:57.302539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 921/1833] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:57.307155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12fbc5-430b-4c6d-9832-06dd4c8f0d5f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:57.320273 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12fbc5-430b-4c6d-9832-06dd4c8f0d5f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:57.326587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe75d2-1953-4205-9b15-432f7c3c5044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:57.327437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe75d2-1953-4205-9b15-432f7c3c5044 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:57.327722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe75d2-1953-4205-9b15-432f7c3c5044 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:57.327999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe75d2-1953-4205-9b15-432f7c3c5044 admin admin] PATH_INFO: `/v3/projects/520e151d33ca450ab8a690774d6715cb/users/e20e467a63ea4d788578c880789a48b7/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:57.331370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe75d2-1953-4205-9b15-432f7c3c5044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=520e151d33ca450ab8a690774d6715cb, user_id=e20e467a63ea4d788578c880789a48b7, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:57.334249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe75d2-1953-4205-9b15-432f7c3c5044 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:57.349252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 913/1834] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:57 2022] PUT /identity/v3/projects/520e151d33ca450ab8a690774d6715cb/users/e20e467a63ea4d788578c880789a48b7/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:57.354762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dad1d-f45f-463c-9bf6-0c310c78ea49 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:57.357632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-114dad1d-f45f-463c-9bf6-0c310c78ea49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:57.360705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbc315-77e2-4a24-a7bd-3e0dcc6ddd08 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:57.361373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbc315-77e2-4a24-a7bd-3e0dcc6ddd08 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:57.362218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbc315-77e2-4a24-a7bd-3e0dcc6ddd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:57.380632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dad1d-f45f-463c-9bf6-0c310c78ea49 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:57.393758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:57.416191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4dcbc315-77e2-4a24-a7bd-3e0dcc6ddd08 None None] MFA Rules not processed for user `e20e467a63ea4d788578c880789a48b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:57.418072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4dcbc315-77e2-4a24-a7bd-3e0dcc6ddd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:57.423381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:57.476457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:57.477497 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:57.477935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:57.478112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] PATH_INFO: `/v3/projects/0da74ec711dd4d2e85e8f970f4c5a93e` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:57.479931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=0da74ec711dd4d2e85e8f970f4c5a93e)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:57.486049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:57.534171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 914/1835] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:54:57.551917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbc315-77e2-4a24-a7bd-3e0dcc6ddd08 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:57.553876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4dcbc315-77e2-4a24-a7bd-3e0dcc6ddd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:57.645044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0da74ec711dd4d2e85e8f970f4c5a93e', 'request_id': 'req-77b14c32-dfb0-4914-a734-144cdee523ea'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:57.679163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0da74ec711dd4d2e85e8f970f4c5a93e', 'request_id': 'req-77b14c32-dfb0-4914-a734-144cdee523ea'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:57.705497 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0da74ec711dd4d2e85e8f970f4c5a93e', 'request_id': 'req-77b14c32-dfb0-4914-a734-144cdee523ea'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:57.739587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] The token cache is being invalidate because project 0da74ec711dd4d2e85e8f970f4c5a93e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:54:57.740198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:57.741863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-77b14c32-dfb0-4914-a734-144cdee523ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:54:57.755829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 922/1836] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:54:57 2022] DELETE /identity/v3/projects/0da74ec711dd4d2e85e8f970f4c5a93e => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:57.758341 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb0353-96c5-44c9-af01-754a6af3019d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:57.758923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb0353-96c5-44c9-af01-754a6af3019d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:57.759494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb0353-96c5-44c9-af01-754a6af3019d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:57.811230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e5eb0353-96c5-44c9-af01-754a6af3019d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:57.814577 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e5eb0353-96c5-44c9-af01-754a6af3019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:57.821233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbc315-77e2-4a24-a7bd-3e0dcc6ddd08 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:58.013636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf98df-320f-4930-b4f4-2abddbe981cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:58.015553 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf98df-320f-4930-b4f4-2abddbe981cd service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:58.015923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf98df-320f-4930-b4f4-2abddbe981cd service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:58.017277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf98df-320f-4930-b4f4-2abddbe981cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:58.019822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2cbf98df-320f-4930-b4f4-2abddbe981cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:58.126930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 923/1837] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:58.135227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa843b2-9430-4a01-8fee-e44b92e9eabe None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:58.135824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa843b2-9430-4a01-8fee-e44b92e9eabe None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:58.136221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa843b2-9430-4a01-8fee-e44b92e9eabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:58.190118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-daa843b2-9430-4a01-8fee-e44b92e9eabe None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:58.192510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-daa843b2-9430-4a01-8fee-e44b92e9eabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:58.223909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 924/1838] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:54:58.225644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa843b2-9430-4a01-8fee-e44b92e9eabe None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:58.231005 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf98df-320f-4930-b4f4-2abddbe981cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:58.233477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf98df-320f-4930-b4f4-2abddbe981cd service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:58.251583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa843b2-9430-4a01-8fee-e44b92e9eabe None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:58.259813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d57a0-e9b4-44a9-917a-81a3956be441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:58.260931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 915/1839] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:54:58.261636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d57a0-e9b4-44a9-917a-81a3956be441 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:58.262103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d57a0-e9b4-44a9-917a-81a3956be441 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:58.262551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d57a0-e9b4-44a9-917a-81a3956be441 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:58.263496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d57a0-e9b4-44a9-917a-81a3956be441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:54:58.264103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d57a0-e9b4-44a9-917a-81a3956be441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:58.266425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d57a0-e9b4-44a9-917a-81a3956be441 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:58.284097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 925/1840] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:54:58.288700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf98df-320f-4930-b4f4-2abddbe981cd service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:58.303333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf98df-320f-4930-b4f4-2abddbe981cd service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:58.316850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5f93f-6d93-47f3-a820-daf9afb1a9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:58.319820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5f93f-6d93-47f3-a820-daf9afb1a9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:58.323233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5f93f-6d93-47f3-a820-daf9afb1a9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:58.323773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5f93f-6d93-47f3-a820-daf9afb1a9a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:58.324718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5f93f-6d93-47f3-a820-daf9afb1a9a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:58.326908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5f93f-6d93-47f3-a820-daf9afb1a9a1 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:58.343600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:58.462101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d57a0-e9b4-44a9-917a-81a3956be441 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:58.465209 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-503d57a0-e9b4-44a9-917a-81a3956be441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:58.481913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 916/1841] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:58 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:58.483395 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5f93f-6d93-47f3-a820-daf9afb1a9a1 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:58.528522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d57a0-e9b4-44a9-917a-81a3956be441 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:58.600846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:58.656388 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2ed7e-4fea-41b1-9c93-996aea4d92ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:58.657876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2ed7e-4fea-41b1-9c93-996aea4d92ac service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:58.657876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2ed7e-4fea-41b1-9c93-996aea4d92ac service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:58.657876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2ed7e-4fea-41b1-9c93-996aea4d92ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:58.659827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-36d2ed7e-4fea-41b1-9c93-996aea4d92ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:58.667337 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5f93f-6d93-47f3-a820-daf9afb1a9a1 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:58.681402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97026a-fb5b-4c1e-88e6-58da36383bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:58.682762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97026a-fb5b-4c1e-88e6-58da36383bea admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:58.683677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97026a-fb5b-4c1e-88e6-58da36383bea admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:58.684301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97026a-fb5b-4c1e-88e6-58da36383bea admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:58.686415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97026a-fb5b-4c1e-88e6-58da36383bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:58.696365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97026a-fb5b-4c1e-88e6-58da36383bea admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:58.753752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 917/1842] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:58 2022] POST /identity/v3/users => generated 291 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:58.757105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97026a-fb5b-4c1e-88e6-58da36383bea admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:58.787086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97026a-fb5b-4c1e-88e6-58da36383bea admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:58.801969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1440d28-7b0e-421d-9401-aed8edaf1252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:58.803649 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1440d28-7b0e-421d-9401-aed8edaf1252 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:58.804109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1440d28-7b0e-421d-9401-aed8edaf1252 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:58.804691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1440d28-7b0e-421d-9401-aed8edaf1252 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:58.806398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1440d28-7b0e-421d-9401-aed8edaf1252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:58.809888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1440d28-7b0e-421d-9401-aed8edaf1252 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:58.827621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2ed7e-4fea-41b1-9c93-996aea4d92ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:58.835108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2ed7e-4fea-41b1-9c93-996aea4d92ac service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:58.859521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 918/1843] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:58.860900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1440d28-7b0e-421d-9401-aed8edaf1252 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:58.861568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 926/1844] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:54:58.898106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1440d28-7b0e-421d-9401-aed8edaf1252 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:58.909541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c40b8-37d4-44c2-bc58-e3202d1759da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:58.910230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c40b8-37d4-44c2-bc58-e3202d1759da admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:58.910519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c40b8-37d4-44c2-bc58-e3202d1759da admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:58.910800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c40b8-37d4-44c2-bc58-e3202d1759da admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:58.911543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c40b8-37d4-44c2-bc58-e3202d1759da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:58.917797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c40b8-37d4-44c2-bc58-e3202d1759da admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:58.950899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 919/1845] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:54:58.955781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2ed7e-4fea-41b1-9c93-996aea4d92ac service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:58.984108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2ed7e-4fea-41b1-9c93-996aea4d92ac service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:58.998908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c6955-6caa-4aa5-9978-844863f9c8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:59.001052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c6955-6caa-4aa5-9978-844863f9c8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:59.001357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c6955-6caa-4aa5-9978-844863f9c8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:59.001710 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c6955-6caa-4aa5-9978-844863f9c8b6 admin admin] PATH_INFO: `/v3/projects/1e3596a8d2374574ac367de5fea20c8d/users/12b4104327fd4445a13a5ef482bb7111/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:59.023804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c6955-6caa-4aa5-9978-844863f9c8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3596a8d2374574ac367de5fea20c8d, user_id=12b4104327fd4445a13a5ef482bb7111, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:59.027294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c6955-6caa-4aa5-9978-844863f9c8b6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:59.056857 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 927/1846] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:54:58 2022] PUT /identity/v3/projects/1e3596a8d2374574ac367de5fea20c8d/users/12b4104327fd4445a13a5ef482bb7111/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:54:59.102131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407844d-0948-41ab-9abf-b1dd0a30e462 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:59.102519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407844d-0948-41ab-9abf-b1dd0a30e462 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:59.103346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407844d-0948-41ab-9abf-b1dd0a30e462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:59.179651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-5407844d-0948-41ab-9abf-b1dd0a30e462 None None] MFA Rules not processed for user `12b4104327fd4445a13a5ef482bb7111`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:59.179651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-5407844d-0948-41ab-9abf-b1dd0a30e462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:59.475221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 920/1847] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:54:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:54:59.497189 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c6955-6caa-4aa5-9978-844863f9c8b6 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:59.497808 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab5fb8-0e0a-4f16-bc8b-88fd6012696d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:59.497808 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab5fb8-0e0a-4f16-bc8b-88fd6012696d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:59.497808 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab5fb8-0e0a-4f16-bc8b-88fd6012696d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:59.540942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3fab5fb8-0e0a-4f16-bc8b-88fd6012696d None None] MFA Rules not processed for user `5db827b3a66843749e04376559aa27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:54:59.542449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3fab5fb8-0e0a-4f16-bc8b-88fd6012696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:54:59.676203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c6955-6caa-4aa5-9978-844863f9c8b6 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:59.681789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531508-d55d-4ed5-9006-3f2ee4bf6591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:59.682624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531508-d55d-4ed5-9006-3f2ee4bf6591 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:59.682910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531508-d55d-4ed5-9006-3f2ee4bf6591 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:59.683532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:54:59.684795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea531508-d55d-4ed5-9006-3f2ee4bf6591 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:59.685850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea531508-d55d-4ed5-9006-3f2ee4bf6591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:59.689439 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea531508-d55d-4ed5-9006-3f2ee4bf6591 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:59.727362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 921/1848] 10.222.0.94 () {58 vars in 1015 bytes} [Tue Jul 19 20:54:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:54:59.757739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 928/1849] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:54:59 2022] POST /identity/v3/projects => generated 381 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:54:59.760886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab5fb8-0e0a-4f16-bc8b-88fd6012696d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:59.781024 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab5fb8-0e0a-4f16-bc8b-88fd6012696d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:54:59.962257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe8687-7658-4a08-b1e3-8cc205860e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:54:59.963853 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe8687-7658-4a08-b1e3-8cc205860e5d admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:54:59.964654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe8687-7658-4a08-b1e3-8cc205860e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:54:59.965426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe8687-7658-4a08-b1e3-8cc205860e5d admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:54:59.967648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe8687-7658-4a08-b1e3-8cc205860e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:54:59.990884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe8687-7658-4a08-b1e3-8cc205860e5d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:54:59.992569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531508-d55d-4ed5-9006-3f2ee4bf6591 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:54:59.994548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ea531508-d55d-4ed5-9006-3f2ee4bf6591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:00.048082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 922/1850] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:54:59 2022] POST /identity/v3/users => generated 297 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:55:00.049843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe8687-7658-4a08-b1e3-8cc205860e5d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:00.056035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dcbe8687-7658-4a08-b1e3-8cc205860e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:00.144313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:00.184549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea531508-d55d-4ed5-9006-3f2ee4bf6591 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:00.194228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bdab5-d391-4cb4-a2b9-cb8d800687d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:00.206205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe8687-7658-4a08-b1e3-8cc205860e5d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:00.209266 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bdab5-d391-4cb4-a2b9-cb8d800687d8 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:00.210013 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bdab5-d391-4cb4-a2b9-cb8d800687d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:00.211892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bdab5-d391-4cb4-a2b9-cb8d800687d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:00.214869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059c714-f75f-408d-a10c-d77595c59f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:00.215675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059c714-f75f-408d-a10c-d77595c59f60 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:00.215961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059c714-f75f-408d-a10c-d77595c59f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:00.216244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059c714-f75f-408d-a10c-d77595c59f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:00.234726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059c714-f75f-408d-a10c-d77595c59f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:00.237291 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059c714-f75f-408d-a10c-d77595c59f60 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:00.245867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bdab5-d391-4cb4-a2b9-cb8d800687d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:00.251101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bdab5-d391-4cb4-a2b9-cb8d800687d8 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:00.256760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 923/1851] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:00.259502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059c714-f75f-408d-a10c-d77595c59f60 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:00.261496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f059c714-f75f-408d-a10c-d77595c59f60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:00.283103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059c714-f75f-408d-a10c-d77595c59f60 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:00.291991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4737a5c-9882-4649-899e-ae65bafa06ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:00.293107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4737a5c-9882-4649-899e-ae65bafa06ef service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:00.293577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4737a5c-9882-4649-899e-ae65bafa06ef service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:00.294067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4737a5c-9882-4649-899e-ae65bafa06ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:00.294803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 929/1852] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:54:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:00.296653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bdab5-d391-4cb4-a2b9-cb8d800687d8 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:00.316660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4737a5c-9882-4649-899e-ae65bafa06ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:00.318986 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4737a5c-9882-4649-899e-ae65bafa06ef service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:00.335342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 924/1853] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:00.349562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bdab5-d391-4cb4-a2b9-cb8d800687d8 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:00.368716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e46c6-c53b-4281-8a95-fca2977a09ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:00.369547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e46c6-c53b-4281-8a95-fca2977a09ba admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:00.369878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e46c6-c53b-4281-8a95-fca2977a09ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:00.370156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e46c6-c53b-4281-8a95-fca2977a09ba admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:00.370971 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e46c6-c53b-4281-8a95-fca2977a09ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:00.377630 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e46c6-c53b-4281-8a95-fca2977a09ba admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:00.415846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4737a5c-9882-4649-899e-ae65bafa06ef service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:00.418109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 930/1854] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:00.423943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e46c6-c53b-4281-8a95-fca2977a09ba admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:00.449294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e46c6-c53b-4281-8a95-fca2977a09ba admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:00.462277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0570633-8417-425a-acf1-feab06b20374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:00.463728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0570633-8417-425a-acf1-feab06b20374 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:00.464280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0570633-8417-425a-acf1-feab06b20374 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:00.464802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0570633-8417-425a-acf1-feab06b20374 admin admin] PATH_INFO: `/v3/projects/d310a51fb26c4a0f9dd6374460a16b63/users/8ed66ba2580a4b9d93813dac3a256da8/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:00.511654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0570633-8417-425a-acf1-feab06b20374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d310a51fb26c4a0f9dd6374460a16b63, user_id=8ed66ba2580a4b9d93813dac3a256da8, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:00.515867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0570633-8417-425a-acf1-feab06b20374 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:00.542319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 931/1855] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:00 2022] PUT /identity/v3/projects/d310a51fb26c4a0f9dd6374460a16b63/users/8ed66ba2580a4b9d93813dac3a256da8/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:00.546530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0570633-8417-425a-acf1-feab06b20374 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:00.665582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4737a5c-9882-4649-899e-ae65bafa06ef service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:00.741391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:00.823170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5295fd-56e5-40bd-989c-71bc05f7b334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:00.823862 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5295fd-56e5-40bd-989c-71bc05f7b334 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:00.824168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5295fd-56e5-40bd-989c-71bc05f7b334 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:00.824639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5295fd-56e5-40bd-989c-71bc05f7b334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:00.826721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dd5295fd-56e5-40bd-989c-71bc05f7b334 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:00.886818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0570633-8417-425a-acf1-feab06b20374 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:00.930804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2569f86-de01-4715-95ba-dc77143636dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:00.934203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2569f86-de01-4715-95ba-dc77143636dd admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:00.934554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2569f86-de01-4715-95ba-dc77143636dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:00.934852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2569f86-de01-4715-95ba-dc77143636dd admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:00.935655 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2569f86-de01-4715-95ba-dc77143636dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:00.954603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2569f86-de01-4715-95ba-dc77143636dd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:01.008189 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 932/1856] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:55:01.010087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2569f86-de01-4715-95ba-dc77143636dd admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:01.057830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2569f86-de01-4715-95ba-dc77143636dd admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:01.064246 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5295fd-56e5-40bd-989c-71bc05f7b334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:01.065912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5295fd-56e5-40bd-989c-71bc05f7b334 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:01.086604 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0942fd-e7b6-4c7d-9798-eed7b90495a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:01.087526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0942fd-e7b6-4c7d-9798-eed7b90495a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:01.087810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0942fd-e7b6-4c7d-9798-eed7b90495a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:01.088087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0942fd-e7b6-4c7d-9798-eed7b90495a3 admin admin] PATH_INFO: `/v3/projects/d310a51fb26c4a0f9dd6374460a16b63/users/8ed66ba2580a4b9d93813dac3a256da8/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:01.094151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 925/1857] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:01.111874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0942fd-e7b6-4c7d-9798-eed7b90495a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d310a51fb26c4a0f9dd6374460a16b63, user_id=8ed66ba2580a4b9d93813dac3a256da8, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:01.115769 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0942fd-e7b6-4c7d-9798-eed7b90495a3 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:01.155601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 933/1858] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:01 2022] PUT /identity/v3/projects/d310a51fb26c4a0f9dd6374460a16b63/users/8ed66ba2580a4b9d93813dac3a256da8/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:01.159138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce1557-66c9-4610-aae3-02d93edaa98d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:01.160303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce1557-66c9-4610-aae3-02d93edaa98d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:01.161064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce1557-66c9-4610-aae3-02d93edaa98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:01.259926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-85ce1557-66c9-4610-aae3-02d93edaa98d None None] MFA Rules not processed for user `8ed66ba2580a4b9d93813dac3a256da8`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:01.268129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-85ce1557-66c9-4610-aae3-02d93edaa98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:01.551650 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 926/1859] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:01.947620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0942fd-e7b6-4c7d-9798-eed7b90495a3 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:01.951870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-bb0942fd-e7b6-4c7d-9798-eed7b90495a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:02.067617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0942fd-e7b6-4c7d-9798-eed7b90495a3 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:02.075394 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6db8a4-ff5f-45bd-b1a3-5114e19ac047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:02.077803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6db8a4-ff5f-45bd-b1a3-5114e19ac047 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:02.078321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6db8a4-ff5f-45bd-b1a3-5114e19ac047 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:02.079624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6db8a4-ff5f-45bd-b1a3-5114e19ac047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:02.129383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6db8a4-ff5f-45bd-b1a3-5114e19ac047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:02.131013 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6db8a4-ff5f-45bd-b1a3-5114e19ac047 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:02.195390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 934/1860] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:03.119029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce1557-66c9-4610-aae3-02d93edaa98d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:03.133028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce1557-66c9-4610-aae3-02d93edaa98d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:03.146624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25269d0f-4b44-4016-902e-93407083abba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:03.147822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25269d0f-4b44-4016-902e-93407083abba service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:03.149899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25269d0f-4b44-4016-902e-93407083abba service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:03.150555 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25269d0f-4b44-4016-902e-93407083abba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:03.356772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25269d0f-4b44-4016-902e-93407083abba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:03.360518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25269d0f-4b44-4016-902e-93407083abba service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:03.402167 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 927/1861] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:03.759925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad1e4a-41fc-4c8e-8e85-f3f0f7a48973 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:03.760902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad1e4a-41fc-4c8e-8e85-f3f0f7a48973 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:03.764471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad1e4a-41fc-4c8e-8e85-f3f0f7a48973 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:03.771591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 935/1862] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 20:55:03 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:55:03.794520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9094e-782e-49b5-9ac5-0d140e5c119b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:03.795394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9094e-782e-49b5-9ac5-0d140e5c119b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:03.795828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9094e-782e-49b5-9ac5-0d140e5c119b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:03.819606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:03.882630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-72f9094e-782e-49b5-9ac5-0d140e5c119b None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:03.887797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-72f9094e-782e-49b5-9ac5-0d140e5c119b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:03.921323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad1e4a-41fc-4c8e-8e85-f3f0f7a48973 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:03.961950 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad1e4a-41fc-4c8e-8e85-f3f0f7a48973 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:03.981066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a226093-8d13-4bc8-b6da-b802bc948fee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:03.981875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226093-8d13-4bc8-b6da-b802bc948fee service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:03.982280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226093-8d13-4bc8-b6da-b802bc948fee service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:03.982606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a226093-8d13-4bc8-b6da-b802bc948fee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:03.994661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9a226093-8d13-4bc8-b6da-b802bc948fee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:04.188972 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:04.285173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 928/1863] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:04.301995 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f5af7-2049-4a71-afba-c5b3c54c7f81 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:04.302425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f5af7-2049-4a71-afba-c5b3c54c7f81 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:04.302766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f5af7-2049-4a71-afba-c5b3c54c7f81 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:04.313472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 929/1864] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 20:55:04 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:55:04.354641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f5af7-2049-4a71-afba-c5b3c54c7f81 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:04.356587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-eb2f5af7-2049-4a71-afba-c5b3c54c7f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:04.364272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a226093-8d13-4bc8-b6da-b802bc948fee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:04.366049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a226093-8d13-4bc8-b6da-b802bc948fee service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:04.444175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f5af7-2049-4a71-afba-c5b3c54c7f81 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:04.446070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 936/1865] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:55:04.474829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059012ce-6cf0-40dd-84d7-5d4b10b37df1 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:04.476106 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059012ce-6cf0-40dd-84d7-5d4b10b37df1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:04.481169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059012ce-6cf0-40dd-84d7-5d4b10b37df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:04.492258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc33ee-2eb6-4aea-87fa-6065e10b5fb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:04.493003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc33ee-2eb6-4aea-87fa-6065e10b5fb6 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:04.493312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc33ee-2eb6-4aea-87fa-6065e10b5fb6 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:04.493614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc33ee-2eb6-4aea-87fa-6065e10b5fb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:04.545986 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc33ee-2eb6-4aea-87fa-6065e10b5fb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:04.554618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc33ee-2eb6-4aea-87fa-6065e10b5fb6 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:04.571807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-059012ce-6cf0-40dd-84d7-5d4b10b37df1 None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:04.571807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-059012ce-6cf0-40dd-84d7-5d4b10b37df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:04.587817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 930/1866] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:04.631666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 937/1867] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 20:55:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:04.843645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc33ee-2eb6-4aea-87fa-6065e10b5fb6 service nova] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:04.847893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-9abc33ee-2eb6-4aea-87fa-6065e10b5fb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:05.057496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc33ee-2eb6-4aea-87fa-6065e10b5fb6 service nova] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:05.065355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:05.072455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:05.074224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:05.075835 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] PATH_INFO: `/v3/users/11a86fb35d244359a9e7418f93309cef` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:05.080117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] RBAC: Authorizing `identity:delete_user(user_id=11a86fb35d244359a9e7418f93309cef)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:05.095346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:05.238874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11a86fb35d244359a9e7418f93309cef'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:05.270974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11a86fb35d244359a9e7418f93309cef'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:05.298227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11a86fb35d244359a9e7418f93309cef'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:05.305717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11a86fb35d244359a9e7418f93309cef'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:05.377086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11a86fb35d244359a9e7418f93309cef'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:05.406391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 931/1868] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:04 2022] DELETE /identity/v3/users/11a86fb35d244359a9e7418f93309cef => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:05.409505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026a2a5-dac4-4760-b0a0-aa8aa2a0b810 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:05.410412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026a2a5-dac4-4760-b0a0-aa8aa2a0b810 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:05.411514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026a2a5-dac4-4760-b0a0-aa8aa2a0b810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:05.472795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0026a2a5-dac4-4760-b0a0-aa8aa2a0b810 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:05.472795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0026a2a5-dac4-4760-b0a0-aa8aa2a0b810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:05.692754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 938/1869] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:05.727833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:05.738243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:05.754357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:05.784229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026a2a5-dac4-4760-b0a0-aa8aa2a0b810 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:05.786055 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0026a2a5-dac4-4760-b0a0-aa8aa2a0b810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:06.014793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac77bf-3d2b-4c1e-a8ce-3ebbef6f936e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:06.028519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab195c6-f604-445c-9a29-1b4f6ce7d1b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:06.030938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab195c6-f604-445c-9a29-1b4f6ce7d1b8 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:06.031415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab195c6-f604-445c-9a29-1b4f6ce7d1b8 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:06.031825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab195c6-f604-445c-9a29-1b4f6ce7d1b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:06.034500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dab195c6-f604-445c-9a29-1b4f6ce7d1b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:06.057540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab195c6-f604-445c-9a29-1b4f6ce7d1b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:06.060388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab195c6-f604-445c-9a29-1b4f6ce7d1b8 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:06.094278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 932/1870] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:06.119230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026a2a5-dac4-4760-b0a0-aa8aa2a0b810 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:06.127169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81ce26-3a8d-4962-a7b8-4542d4ba5fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:06.128148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81ce26-3a8d-4962-a7b8-4542d4ba5fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:06.128634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81ce26-3a8d-4962-a7b8-4542d4ba5fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:06.129049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81ce26-3a8d-4962-a7b8-4542d4ba5fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:06.156982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81ce26-3a8d-4962-a7b8-4542d4ba5fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:06.165922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81ce26-3a8d-4962-a7b8-4542d4ba5fa2 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:06.214234 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 939/1871] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:07.410729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab195c6-f604-445c-9a29-1b4f6ce7d1b8 service nova] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:07.412752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dab195c6-f604-445c-9a29-1b4f6ce7d1b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:07.452918 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab195c6-f604-445c-9a29-1b4f6ce7d1b8 service nova] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:07.470740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:07.474210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:07.474539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:07.477702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] PATH_INFO: `/v3/users/12b4104327fd4445a13a5ef482bb7111` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:07.486817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b4104327fd4445a13a5ef482bb7111)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:07.500112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:07.711950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b4104327fd4445a13a5ef482bb7111'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:07.748988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b4104327fd4445a13a5ef482bb7111'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:07.765244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b4104327fd4445a13a5ef482bb7111'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:07.766855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b4104327fd4445a13a5ef482bb7111'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:07.825970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b4104327fd4445a13a5ef482bb7111'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:07.863596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 933/1872] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:07 2022] DELETE /identity/v3/users/12b4104327fd4445a13a5ef482bb7111 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:07.871348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61766eeb-7a99-433c-a5db-89174bfb9750 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:07.872284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61766eeb-7a99-433c-a5db-89174bfb9750 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:07.872744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61766eeb-7a99-433c-a5db-89174bfb9750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:07.955979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-61766eeb-7a99-433c-a5db-89174bfb9750 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:07.957924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-61766eeb-7a99-433c-a5db-89174bfb9750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:08.161557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:08.163335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:08.182378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 940/1873] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:08.217757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61766eeb-7a99-433c-a5db-89174bfb9750 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:08.220464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-61766eeb-7a99-433c-a5db-89174bfb9750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:08.243756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:08.365804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4c354-a4ad-43da-bd04-539b7a6c1e76 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:08.371421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:08.372314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:08.372629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:08.373084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] PATH_INFO: `/v3/projects/63853b37948843a28c082110f29be672` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:08.374398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=63853b37948843a28c082110f29be672)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:08.377137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:08.378547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:08.393631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61766eeb-7a99-433c-a5db-89174bfb9750 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:08.406040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef90d4e-3b94-47ed-b896-8a387729b571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:08.406988 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef90d4e-3b94-47ed-b896-8a387729b571 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:08.407458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef90d4e-3b94-47ed-b896-8a387729b571 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:08.407977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef90d4e-3b94-47ed-b896-8a387729b571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:08.455446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef90d4e-3b94-47ed-b896-8a387729b571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:08.457371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef90d4e-3b94-47ed-b896-8a387729b571 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:08.480949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63853b37948843a28c082110f29be672', 'request_id': 'req-4982794d-272c-46a5-9ae3-e3bf2067aa6f'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:08.483951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 941/1874] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:08.492778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63853b37948843a28c082110f29be672', 'request_id': 'req-4982794d-272c-46a5-9ae3-e3bf2067aa6f'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:08.508262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63853b37948843a28c082110f29be672', 'request_id': 'req-4982794d-272c-46a5-9ae3-e3bf2067aa6f'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:08.536887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] The token cache is being invalidate because project 63853b37948843a28c082110f29be672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:08.537186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:08.538669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:08.540406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 934/1875] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:08 2022] DELETE /identity/v3/projects/63853b37948843a28c082110f29be672 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:08.545172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef90d4e-3b94-47ed-b896-8a387729b571 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:08.562398 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef90d4e-3b94-47ed-b896-8a387729b571 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:08.687725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:08.689513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:08.692750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:08.693042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] PATH_INFO: `/v3/users/e20e467a63ea4d788578c880789a48b7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:08.702367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e20e467a63ea4d788578c880789a48b7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:08.705130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:08.797183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e20e467a63ea4d788578c880789a48b7'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:08.807656 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e20e467a63ea4d788578c880789a48b7'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:08.808481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e20e467a63ea4d788578c880789a48b7'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:08.838583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e20e467a63ea4d788578c880789a48b7'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:08.848929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e20e467a63ea4d788578c880789a48b7'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:08.857992 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 942/1876] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:08 2022] DELETE /identity/v3/users/e20e467a63ea4d788578c880789a48b7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:09.224012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:09.225539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:09.241332 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982794d-272c-46a5-9ae3-e3bf2067aa6f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:09.341845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:09.343608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:09.344303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:09.344836 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] PATH_INFO: `/v3/projects/1e3596a8d2374574ac367de5fea20c8d` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:09.349210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3596a8d2374574ac367de5fea20c8d)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:09.352435 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:09.382321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:09.384175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:09.400412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63266d6-524e-4aba-9021-c22b2ef078d6 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:09.407712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:09.408795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:09.409116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:09.409366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] PATH_INFO: `/v3/projects/520e151d33ca450ab8a690774d6715cb` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:09.410822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=520e151d33ca450ab8a690774d6715cb)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:09.413247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:09.457214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3596a8d2374574ac367de5fea20c8d', 'request_id': 'req-588ad2c3-0c0e-477b-b44b-42eed2dd066d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.478675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:09.483663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3596a8d2374574ac367de5fea20c8d', 'request_id': 'req-588ad2c3-0c0e-477b-b44b-42eed2dd066d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.504655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3596a8d2374574ac367de5fea20c8d', 'request_id': 'req-588ad2c3-0c0e-477b-b44b-42eed2dd066d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.526516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '520e151d33ca450ab8a690774d6715cb', 'request_id': 'req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.537263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] The token cache is being invalidate because project 1e3596a8d2374574ac367de5fea20c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:09.538632 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.539394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.541514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 935/1877] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:09 2022] DELETE /identity/v3/projects/1e3596a8d2374574ac367de5fea20c8d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:09.545322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:09.546961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:09.555582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '520e151d33ca450ab8a690774d6715cb', 'request_id': 'req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.576092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '520e151d33ca450ab8a690774d6715cb', 'request_id': 'req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.603605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] The token cache is being invalidate because project 520e151d33ca450ab8a690774d6715cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:09.603898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.604647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-1eda1ccc-3c1a-453f-ae24-3aefea8c8d97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.606149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:09.615705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 943/1878] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:09 2022] DELETE /identity/v3/projects/520e151d33ca450ab8a690774d6715cb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:09.624256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bfa30-78d4-455c-93a8-2d6435a06f77 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:09.624682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bfa30-78d4-455c-93a8-2d6435a06f77 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:09.625049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bfa30-78d4-455c-93a8-2d6435a06f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:09.650171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-d73bfa30-78d4-455c-93a8-2d6435a06f77 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:09.651410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d73bfa30-78d4-455c-93a8-2d6435a06f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:09.659346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ad2c3-0c0e-477b-b44b-42eed2dd066d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:09.724357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 944/1879] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:09.731234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:09.732711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:09.733093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:09.733374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086602be-227b-4cf5-b8fe-43fa42e25cf5 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:09.733478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] PATH_INFO: `/v3/users/8ed66ba2580a4b9d93813dac3a256da8` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:09.733701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086602be-227b-4cf5-b8fe-43fa42e25cf5 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:09.734001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086602be-227b-4cf5-b8fe-43fa42e25cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:09.735169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ed66ba2580a4b9d93813dac3a256da8)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:09.737553 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:09.753402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-086602be-227b-4cf5-b8fe-43fa42e25cf5 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:09.754628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-086602be-227b-4cf5-b8fe-43fa42e25cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:09.762728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 945/1880] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:09.768355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086602be-227b-4cf5-b8fe-43fa42e25cf5 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:09.778696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086602be-227b-4cf5-b8fe-43fa42e25cf5 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:09.783651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a092c-1525-42ae-b4bc-3c8b648a59a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:09.784312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a092c-1525-42ae-b4bc-3c8b648a59a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:09.784620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a092c-1525-42ae-b4bc-3c8b648a59a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:09.784901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a092c-1525-42ae-b4bc-3c8b648a59a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:09.785662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a092c-1525-42ae-b4bc-3c8b648a59a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:09.785968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a092c-1525-42ae-b4bc-3c8b648a59a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:09.787560 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a092c-1525-42ae-b4bc-3c8b648a59a4 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:09.788896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ed66ba2580a4b9d93813dac3a256da8'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.798093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ed66ba2580a4b9d93813dac3a256da8'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.802017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 946/1881] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:55:09.806315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a092c-1525-42ae-b4bc-3c8b648a59a4 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:09.807386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ed66ba2580a4b9d93813dac3a256da8'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.808131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ed66ba2580a4b9d93813dac3a256da8'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.817740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a092c-1525-42ae-b4bc-3c8b648a59a4 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:09.821757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ed66ba2580a4b9d93813dac3a256da8'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:09.825016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c2b08-0529-4d64-9d6b-828d11c9d508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:09.825966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2b08-0529-4d64-9d6b-828d11c9d508 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:09.826376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2b08-0529-4d64-9d6b-828d11c9d508 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:09.826889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2b08-0529-4d64-9d6b-828d11c9d508 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:09.827690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c2b08-0529-4d64-9d6b-828d11c9d508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:09.828362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 936/1882] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:09 2022] DELETE /identity/v3/users/8ed66ba2580a4b9d93813dac3a256da8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:09.830385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c2b08-0529-4d64-9d6b-828d11c9d508 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:09.868519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 947/1883] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:09 2022] POST /identity/v3/projects => generated 409 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:55:09.872425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:09.874859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:09.934103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:09.959720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701cde2-867c-420d-803c-680bb6905b52 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.034130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3195f-62aa-4ed8-b6c8-fe5cb27a4a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.034688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3195f-62aa-4ed8-b6c8-fe5cb27a4a59 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.035044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3195f-62aa-4ed8-b6c8-fe5cb27a4a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.035513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3195f-62aa-4ed8-b6c8-fe5cb27a4a59 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.036611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3195f-62aa-4ed8-b6c8-fe5cb27a4a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.039120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3195f-62aa-4ed8-b6c8-fe5cb27a4a59 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.080392 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 937/1884] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:09 2022] POST /identity/v3/users => generated 305 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:10.084612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c2b08-0529-4d64-9d6b-828d11c9d508 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.098921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c2b08-0529-4d64-9d6b-828d11c9d508 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.106833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b50e4-c597-433c-8c71-85a4a422c7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.107933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b50e4-c597-433c-8c71-85a4a422c7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.108641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b50e4-c597-433c-8c71-85a4a422c7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.108915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b50e4-c597-433c-8c71-85a4a422c7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.110213 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b50e4-c597-433c-8c71-85a4a422c7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.113114 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b50e4-c597-433c-8c71-85a4a422c7c4 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.127075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 948/1885] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:10.131894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3195f-62aa-4ed8-b6c8-fe5cb27a4a59 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.147276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3195f-62aa-4ed8-b6c8-fe5cb27a4a59 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.154414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72c5f5-dba6-4b96-9c3a-9a3d10626194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.155137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72c5f5-dba6-4b96-9c3a-9a3d10626194 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.155426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72c5f5-dba6-4b96-9c3a-9a3d10626194 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.155747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72c5f5-dba6-4b96-9c3a-9a3d10626194 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.156492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72c5f5-dba6-4b96-9c3a-9a3d10626194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.158731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72c5f5-dba6-4b96-9c3a-9a3d10626194 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.175679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 938/1886] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:10.180404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b50e4-c597-433c-8c71-85a4a422c7c4 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.193117 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b50e4-c597-433c-8c71-85a4a422c7c4 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.199232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d32b7a-75ac-4cf9-bd8e-1d4b96d1c7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.200426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d32b7a-75ac-4cf9-bd8e-1d4b96d1c7c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.200745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d32b7a-75ac-4cf9-bd8e-1d4b96d1c7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.201032 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d32b7a-75ac-4cf9-bd8e-1d4b96d1c7c3 admin admin] PATH_INFO: `/v3/projects/cdafd6a2f8d5417690241ae3d8be1437/users/5ca36333601646fdbd295c6cecf72882/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.215414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72c5f5-dba6-4b96-9c3a-9a3d10626194 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.218879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0c72c5f5-dba6-4b96-9c3a-9a3d10626194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:10.230029 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d32b7a-75ac-4cf9-bd8e-1d4b96d1c7c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdafd6a2f8d5417690241ae3d8be1437, user_id=5ca36333601646fdbd295c6cecf72882, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.233336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d32b7a-75ac-4cf9-bd8e-1d4b96d1c7c3 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.234277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72c5f5-dba6-4b96-9c3a-9a3d10626194 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.244676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.246376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.246376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.246663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] PATH_INFO: `/v3/projects/d310a51fb26c4a0f9dd6374460a16b63` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.248342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d310a51fb26c4a0f9dd6374460a16b63)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.250865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.263112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 949/1887] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:10 2022] PUT /identity/v3/projects/cdafd6a2f8d5417690241ae3d8be1437/users/5ca36333601646fdbd295c6cecf72882/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:10.275304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f11f1-07f6-454d-8186-c95bb3b3d00c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.275648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f11f1-07f6-454d-8186-c95bb3b3d00c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.276550 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f11f1-07f6-454d-8186-c95bb3b3d00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.312115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd310a51fb26c4a0f9dd6374460a16b63', 'request_id': 'req-ee396871-b4e4-4f97-a361-4b0e1fb3112b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:10.319120 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-6a3f11f1-07f6-454d-8186-c95bb3b3d00c None None] MFA Rules not processed for user `5ca36333601646fdbd295c6cecf72882`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:10.320641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6a3f11f1-07f6-454d-8186-c95bb3b3d00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:10.323022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd310a51fb26c4a0f9dd6374460a16b63', 'request_id': 'req-ee396871-b4e4-4f97-a361-4b0e1fb3112b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:10.332826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd310a51fb26c4a0f9dd6374460a16b63', 'request_id': 'req-ee396871-b4e4-4f97-a361-4b0e1fb3112b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:10.347442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] The token cache is being invalidate because project d310a51fb26c4a0f9dd6374460a16b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:10.347896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:10.348713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-ee396871-b4e4-4f97-a361-4b0e1fb3112b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:10.348863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:10.351472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 939/1888] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:10 2022] DELETE /identity/v3/projects/d310a51fb26c4a0f9dd6374460a16b63 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:10.357220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7162a-624b-4876-a124-099b55b6b9e6 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.357597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7162a-624b-4876-a124-099b55b6b9e6 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.357829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7162a-624b-4876-a124-099b55b6b9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.398452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-cee7162a-624b-4876-a124-099b55b6b9e6 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:10.404259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cee7162a-624b-4876-a124-099b55b6b9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:10.431710 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 950/1889] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:10.435881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f11f1-07f6-454d-8186-c95bb3b3d00c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.437523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6a3f11f1-07f6-454d-8186-c95bb3b3d00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:10.480844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:10.501713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 940/1890] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:10.511123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbeaae-5cac-46e8-b882-e82f27ce8010 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.511489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbeaae-5cac-46e8-b882-e82f27ce8010 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.512251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbeaae-5cac-46e8-b882-e82f27ce8010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.552627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-dadbeaae-5cac-46e8-b882-e82f27ce8010 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:10.554057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dadbeaae-5cac-46e8-b882-e82f27ce8010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:10.558923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f11f1-07f6-454d-8186-c95bb3b3d00c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.564365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 941/1891] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:10.569448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbeaae-5cac-46e8-b882-e82f27ce8010 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.571145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2527d5-1a9d-4e13-9b8a-ebeafc7c46d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.581359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2527d5-1a9d-4e13-9b8a-ebeafc7c46d2 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.581359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2527d5-1a9d-4e13-9b8a-ebeafc7c46d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.581359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2527d5-1a9d-4e13-9b8a-ebeafc7c46d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.581359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2527d5-1a9d-4e13-9b8a-ebeafc7c46d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.581359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2527d5-1a9d-4e13-9b8a-ebeafc7c46d2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.582068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbeaae-5cac-46e8-b882-e82f27ce8010 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.588672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114efa77-f24f-49a7-9a7d-400bae7ed270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.588672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114efa77-f24f-49a7-9a7d-400bae7ed270 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.589101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114efa77-f24f-49a7-9a7d-400bae7ed270 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.589101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114efa77-f24f-49a7-9a7d-400bae7ed270 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.589723 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114efa77-f24f-49a7-9a7d-400bae7ed270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:10.590031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114efa77-f24f-49a7-9a7d-400bae7ed270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.591620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114efa77-f24f-49a7-9a7d-400bae7ed270 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.620467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 942/1892] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:55:10.623401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 951/1893] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:10 2022] POST /identity/v3/projects => generated 407 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:55:10.624796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114efa77-f24f-49a7-9a7d-400bae7ed270 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.627500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2527d5-1a9d-4e13-9b8a-ebeafc7c46d2 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.638347 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114efa77-f24f-49a7-9a7d-400bae7ed270 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.709151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2527d5-1a9d-4e13-9b8a-ebeafc7c46d2 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.716099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87e85c-946a-4722-b9be-a7355e2ff44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.716722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6541cbf-02f0-46fb-a451-e4d6d5e30a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.717372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87e85c-946a-4722-b9be-a7355e2ff44e admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.717711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6541cbf-02f0-46fb-a451-e4d6d5e30a60 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.718112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6541cbf-02f0-46fb-a451-e4d6d5e30a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.718585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6541cbf-02f0-46fb-a451-e4d6d5e30a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.719444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6541cbf-02f0-46fb-a451-e4d6d5e30a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.719634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87e85c-946a-4722-b9be-a7355e2ff44e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.719958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87e85c-946a-4722-b9be-a7355e2ff44e admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.720776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87e85c-946a-4722-b9be-a7355e2ff44e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.722582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6541cbf-02f0-46fb-a451-e4d6d5e30a60 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.723608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87e85c-946a-4722-b9be-a7355e2ff44e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.756532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 952/1894] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:10 2022] POST /identity/v3/users => generated 310 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:55:10.758658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87e85c-946a-4722-b9be-a7355e2ff44e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.767225 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 943/1895] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:10 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:10.768394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6541cbf-02f0-46fb-a451-e4d6d5e30a60 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.771385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87e85c-946a-4722-b9be-a7355e2ff44e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.831157 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:10.865378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6541cbf-02f0-46fb-a451-e4d6d5e30a60 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.871354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a3247-d00b-4f52-8349-762ed30dcf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.872157 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a3247-d00b-4f52-8349-762ed30dcf16 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.872519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a3247-d00b-4f52-8349-762ed30dcf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.872873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a3247-d00b-4f52-8349-762ed30dcf16 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.873452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d842b3-1484-4e03-9b8b-048ffb5bb6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.874184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a3247-d00b-4f52-8349-762ed30dcf16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.876201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d842b3-1484-4e03-9b8b-048ffb5bb6fd admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.876201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d842b3-1484-4e03-9b8b-048ffb5bb6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.876201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d842b3-1484-4e03-9b8b-048ffb5bb6fd admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.876810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a3247-d00b-4f52-8349-762ed30dcf16 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.877374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d842b3-1484-4e03-9b8b-048ffb5bb6fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.879891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d842b3-1484-4e03-9b8b-048ffb5bb6fd admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.891054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 953/1896] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:10.895024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a3247-d00b-4f52-8349-762ed30dcf16 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.908897 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a3247-d00b-4f52-8349-762ed30dcf16 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.911614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 944/1897] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:10 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:10.916247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d842b3-1484-4e03-9b8b-048ffb5bb6fd admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.920908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aa37a-72c7-4700-85a8-cf15c1cb320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.926577 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aa37a-72c7-4700-85a8-cf15c1cb320e admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.927060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aa37a-72c7-4700-85a8-cf15c1cb320e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.927484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aa37a-72c7-4700-85a8-cf15c1cb320e admin admin] PATH_INFO: `/v3/projects/7a8a5d06fbdd46f3adb29440b48ffaaa/users/e0cf74dabfab45428ec0d4c56298d360/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.934967 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d842b3-1484-4e03-9b8b-048ffb5bb6fd admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:10.945358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aed0f-2e36-4147-9b7e-5add46602fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:10.946326 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aed0f-2e36-4147-9b7e-5add46602fdb admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:10.946778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aed0f-2e36-4147-9b7e-5add46602fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:10.947454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aed0f-2e36-4147-9b7e-5add46602fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:10.947966 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aed0f-2e36-4147-9b7e-5add46602fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.950567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aed0f-2e36-4147-9b7e-5add46602fdb admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.951677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353aa37a-72c7-4700-85a8-cf15c1cb320e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8a5d06fbdd46f3adb29440b48ffaaa, user_id=e0cf74dabfab45428ec0d4c56298d360, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:10.955350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353aa37a-72c7-4700-85a8-cf15c1cb320e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:10.971138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 945/1898] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:55:10.975903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aed0f-2e36-4147-9b7e-5add46602fdb admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.986388 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 954/1899] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:10 2022] PUT /identity/v3/projects/7a8a5d06fbdd46f3adb29440b48ffaaa/users/e0cf74dabfab45428ec0d4c56298d360/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:10.993500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aa37a-72c7-4700-85a8-cf15c1cb320e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:10.995692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aed0f-2e36-4147-9b7e-5add46602fdb admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.087873 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:11.128914 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687b5cc-b85b-4be0-87ea-f3d2e662da8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.129706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b5cc-b85b-4be0-87ea-f3d2e662da8a admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.130028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b5cc-b85b-4be0-87ea-f3d2e662da8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.130372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b5cc-b85b-4be0-87ea-f3d2e662da8a admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.131220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687b5cc-b85b-4be0-87ea-f3d2e662da8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.133461 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687b5cc-b85b-4be0-87ea-f3d2e662da8a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.137562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aa37a-72c7-4700-85a8-cf15c1cb320e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.144142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4ad64-2db2-42f1-81db-16f041122131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.145227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4ad64-2db2-42f1-81db-16f041122131 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.145721 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4ad64-2db2-42f1-81db-16f041122131 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.146315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4ad64-2db2-42f1-81db-16f041122131 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.147501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4ad64-2db2-42f1-81db-16f041122131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.149696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4ad64-2db2-42f1-81db-16f041122131 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.152055 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 946/1900] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:11.153397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687b5cc-b85b-4be0-87ea-f3d2e662da8a admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:11.165447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687b5cc-b85b-4be0-87ea-f3d2e662da8a admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.169008 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 955/1901] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:55:11.170763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4ad64-2db2-42f1-81db-16f041122131 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:11.171802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28e469-9540-48f5-9906-c353e7281ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.175053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28e469-9540-48f5-9906-c353e7281ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.175495 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28e469-9540-48f5-9906-c353e7281ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.175925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28e469-9540-48f5-9906-c353e7281ee3 admin admin] PATH_INFO: `/v3/projects/9ff2f24ea6cd44ca95506488ecf993af/users/f7cf632fe24048b59aa5aa59dddc879b/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.184215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4ad64-2db2-42f1-81db-16f041122131 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.193415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06c749-ed56-4952-ad92-c67f4a7089ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.195054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06c749-ed56-4952-ad92-c67f4a7089ee admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.195354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06c749-ed56-4952-ad92-c67f4a7089ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.195636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06c749-ed56-4952-ad92-c67f4a7089ee admin admin] PATH_INFO: `/v3/projects/7a8a5d06fbdd46f3adb29440b48ffaaa/users/e0cf74dabfab45428ec0d4c56298d360/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.199111 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28e469-9540-48f5-9906-c353e7281ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff2f24ea6cd44ca95506488ecf993af, user_id=f7cf632fe24048b59aa5aa59dddc879b, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.199667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06c749-ed56-4952-ad92-c67f4a7089ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8a5d06fbdd46f3adb29440b48ffaaa, user_id=e0cf74dabfab45428ec0d4c56298d360, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.204376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28e469-9540-48f5-9906-c353e7281ee3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.205155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06c749-ed56-4952-ad92-c67f4a7089ee admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.218938 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 956/1902] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:11 2022] PUT /identity/v3/projects/7a8a5d06fbdd46f3adb29440b48ffaaa/users/e0cf74dabfab45428ec0d4c56298d360/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:11.231792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 947/1903] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:11 2022] PUT /identity/v3/projects/9ff2f24ea6cd44ca95506488ecf993af/users/f7cf632fe24048b59aa5aa59dddc879b/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:11.241405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450afab-006f-4782-bd76-e3f660af3cd2 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.242158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450afab-006f-4782-bd76-e3f660af3cd2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.242639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450afab-006f-4782-bd76-e3f660af3cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.244548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631f293-0539-419b-816f-3ab7e6b99f2a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.244548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631f293-0539-419b-816f-3ab7e6b99f2a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.244548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631f293-0539-419b-816f-3ab7e6b99f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.279332 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3631f293-0539-419b-816f-3ab7e6b99f2a None None] MFA Rules not processed for user `f7cf632fe24048b59aa5aa59dddc879b`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:11.280953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3631f293-0539-419b-816f-3ab7e6b99f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:11.302600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e450afab-006f-4782-bd76-e3f660af3cd2 None None] MFA Rules not processed for user `e0cf74dabfab45428ec0d4c56298d360`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:11.304320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e450afab-006f-4782-bd76-e3f660af3cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:11.377980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 948/1904] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:11.382877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631f293-0539-419b-816f-3ab7e6b99f2a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:11.405917 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 957/1905] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:11 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:11.418085 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450afab-006f-4782-bd76-e3f660af3cd2 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:11.420477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e450afab-006f-4782-bd76-e3f660af3cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:11.442824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:11.457426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631f293-0539-419b-816f-3ab7e6b99f2a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.462953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc2bd0-a0fa-4edf-88b0-db7fa0cca1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.463625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc2bd0-a0fa-4edf-88b0-db7fa0cca1db admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.463912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc2bd0-a0fa-4edf-88b0-db7fa0cca1db admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.464199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc2bd0-a0fa-4edf-88b0-db7fa0cca1db admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.464934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc2bd0-a0fa-4edf-88b0-db7fa0cca1db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.466890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc2bd0-a0fa-4edf-88b0-db7fa0cca1db admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.488681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450afab-006f-4782-bd76-e3f660af3cd2 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.492830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662360c-d50c-4dda-928a-b9cdb9a5dbce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.493557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662360c-d50c-4dda-928a-b9cdb9a5dbce service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.493846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662360c-d50c-4dda-928a-b9cdb9a5dbce service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.497212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 949/1906] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:11 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:11.497344 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662360c-d50c-4dda-928a-b9cdb9a5dbce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.498316 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc2bd0-a0fa-4edf-88b0-db7fa0cca1db admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:11.504832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662360c-d50c-4dda-928a-b9cdb9a5dbce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.506409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662360c-d50c-4dda-928a-b9cdb9a5dbce service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.517823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 958/1907] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:11 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:55:11.530659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:11.568101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc2bd0-a0fa-4edf-88b0-db7fa0cca1db admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.574101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e7634-3832-4e0f-ad3a-454631b14660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.574464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e7634-3832-4e0f-ad3a-454631b14660 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.574683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e7634-3832-4e0f-ad3a-454631b14660 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.574928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e7634-3832-4e0f-ad3a-454631b14660 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.575628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e7634-3832-4e0f-ad3a-454631b14660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.577585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e7634-3832-4e0f-ad3a-454631b14660 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.605483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 950/1908] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:11 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:11.609043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662360c-d50c-4dda-928a-b9cdb9a5dbce service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:11.625814 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662360c-d50c-4dda-928a-b9cdb9a5dbce service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.633122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cc254-bb6f-48b6-aa82-6ba3783e0f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.634100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cc254-bb6f-48b6-aa82-6ba3783e0f06 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.634390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cc254-bb6f-48b6-aa82-6ba3783e0f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.634769 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cc254-bb6f-48b6-aa82-6ba3783e0f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.635505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cc254-bb6f-48b6-aa82-6ba3783e0f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.638128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cc254-bb6f-48b6-aa82-6ba3783e0f06 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.653464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 959/1909] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:55:11.658375 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e7634-3832-4e0f-ad3a-454631b14660 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:11.673049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e7634-3832-4e0f-ad3a-454631b14660 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.690178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068dce44-5296-427e-b25f-c4280880159b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.692788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dce44-5296-427e-b25f-c4280880159b admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.692788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dce44-5296-427e-b25f-c4280880159b admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.692788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dce44-5296-427e-b25f-c4280880159b admin admin] PATH_INFO: `/v3/projects/6cbb402bfa134a708699d0301c794fc4/users/6f64c4f84dff4c3dbda2682f63bc55dc/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.718695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068dce44-5296-427e-b25f-c4280880159b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cbb402bfa134a708699d0301c794fc4, user_id=6f64c4f84dff4c3dbda2682f63bc55dc, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.722084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068dce44-5296-427e-b25f-c4280880159b admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.745641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 951/1910] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:11 2022] PUT /identity/v3/projects/6cbb402bfa134a708699d0301c794fc4/users/6f64c4f84dff4c3dbda2682f63bc55dc/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:11.751564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cc254-bb6f-48b6-aa82-6ba3783e0f06 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:11.772633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cc254-bb6f-48b6-aa82-6ba3783e0f06 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.799122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068dce44-5296-427e-b25f-c4280880159b admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:11.863258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893dc50-f83f-42c5-90ad-2d5ec19347a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.864066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893dc50-f83f-42c5-90ad-2d5ec19347a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.864391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893dc50-f83f-42c5-90ad-2d5ec19347a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.864681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893dc50-f83f-42c5-90ad-2d5ec19347a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.865425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893dc50-f83f-42c5-90ad-2d5ec19347a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.867064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893dc50-f83f-42c5-90ad-2d5ec19347a2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.880374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 960/1911] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:11.880374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893dc50-f83f-42c5-90ad-2d5ec19347a2 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:11.881279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068dce44-5296-427e-b25f-c4280880159b admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.883611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-5893dc50-f83f-42c5-90ad-2d5ec19347a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:11.885429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46681ef1-e864-4c1f-8a64-8b1a1181834a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.886150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46681ef1-e864-4c1f-8a64-8b1a1181834a service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.886471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46681ef1-e864-4c1f-8a64-8b1a1181834a service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.886782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46681ef1-e864-4c1f-8a64-8b1a1181834a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.898775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893dc50-f83f-42c5-90ad-2d5ec19347a2 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:11.905499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6196989-07ac-4aca-aa37-d30064f1abb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:11.906772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6196989-07ac-4aca-aa37-d30064f1abb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:11.907129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6196989-07ac-4aca-aa37-d30064f1abb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:11.907722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6196989-07ac-4aca-aa37-d30064f1abb4 admin admin] PATH_INFO: `/v3/users/7a70545a52004b869bbc0aee0cd83950` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:11.909319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6196989-07ac-4aca-aa37-d30064f1abb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a70545a52004b869bbc0aee0cd83950)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.911697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6196989-07ac-4aca-aa37-d30064f1abb4 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.973092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46681ef1-e864-4c1f-8a64-8b1a1181834a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:11.973940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d6196989-07ac-4aca-aa37-d30064f1abb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a70545a52004b869bbc0aee0cd83950'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:11.974580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46681ef1-e864-4c1f-8a64-8b1a1181834a service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:11.985305 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d6196989-07ac-4aca-aa37-d30064f1abb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a70545a52004b869bbc0aee0cd83950'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:11.986017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d6196989-07ac-4aca-aa37-d30064f1abb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a70545a52004b869bbc0aee0cd83950'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:11.986660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 952/1912] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:11 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:11.990480 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46681ef1-e864-4c1f-8a64-8b1a1181834a service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:12.005337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d6196989-07ac-4aca-aa37-d30064f1abb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a70545a52004b869bbc0aee0cd83950'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:12.007229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46681ef1-e864-4c1f-8a64-8b1a1181834a service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:12.015089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4a86b-e052-471a-8a1f-7819c3a0285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:12.016038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d6196989-07ac-4aca-aa37-d30064f1abb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a70545a52004b869bbc0aee0cd83950'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:12.016175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4a86b-e052-471a-8a1f-7819c3a0285f admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:12.016551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4a86b-e052-471a-8a1f-7819c3a0285f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:12.016879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4a86b-e052-471a-8a1f-7819c3a0285f admin admin] PATH_INFO: `/v3/projects/6cbb402bfa134a708699d0301c794fc4/users/6f64c4f84dff4c3dbda2682f63bc55dc/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:12.020081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4a86b-e052-471a-8a1f-7819c3a0285f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cbb402bfa134a708699d0301c794fc4, user_id=6f64c4f84dff4c3dbda2682f63bc55dc, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:12.023775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4a86b-e052-471a-8a1f-7819c3a0285f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:12.024856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 961/1913] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:11 2022] DELETE /identity/v3/users/7a70545a52004b869bbc0aee0cd83950 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:12.029860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ebe1-d1c1-4788-a327-54e6e287616e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:12.030184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ebe1-d1c1-4788-a327-54e6e287616e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:12.030470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90ebe1-d1c1-4788-a327-54e6e287616e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:12.039262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 953/1914] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:11 2022] PUT /identity/v3/projects/6cbb402bfa134a708699d0301c794fc4/users/6f64c4f84dff4c3dbda2682f63bc55dc/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:12.050719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadcd28-661e-482c-9668-13d97f50ad87 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:12.051008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadcd28-661e-482c-9668-13d97f50ad87 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:12.051344 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadcd28-661e-482c-9668-13d97f50ad87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:12.054958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-1e90ebe1-d1c1-4788-a327-54e6e287616e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:12.056643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1e90ebe1-d1c1-4788-a327-54e6e287616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:12.075023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:12.096999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-ccadcd28-661e-482c-9668-13d97f50ad87 None None] MFA Rules not processed for user `6f64c4f84dff4c3dbda2682f63bc55dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:12.098431 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ccadcd28-661e-482c-9668-13d97f50ad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:12.147906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:12.177380 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 962/1915] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:12.189900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90ebe1-d1c1-4788-a327-54e6e287616e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:12.192607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1e90ebe1-d1c1-4788-a327-54e6e287616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:12.277299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 954/1916] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:12.280666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90ebe1-d1c1-4788-a327-54e6e287616e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:12.293238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097b487-5004-4ea8-8d59-680347544699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:12.294124 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097b487-5004-4ea8-8d59-680347544699 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:12.294558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097b487-5004-4ea8-8d59-680347544699 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:12.295086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097b487-5004-4ea8-8d59-680347544699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:12.311324 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097b487-5004-4ea8-8d59-680347544699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:12.315085 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097b487-5004-4ea8-8d59-680347544699 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:12.328429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 963/1917] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:55:12.417585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadcd28-661e-482c-9668-13d97f50ad87 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:12.419824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ccadcd28-661e-482c-9668-13d97f50ad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:12.533949 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadcd28-661e-482c-9668-13d97f50ad87 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:12.539520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc583a1-f558-4764-a1e2-cba9c432e02b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:12.540282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc583a1-f558-4764-a1e2-cba9c432e02b service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:12.540609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc583a1-f558-4764-a1e2-cba9c432e02b service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:12.540892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc583a1-f558-4764-a1e2-cba9c432e02b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:12.553621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc583a1-f558-4764-a1e2-cba9c432e02b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:12.569665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc583a1-f558-4764-a1e2-cba9c432e02b service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:12.597177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 955/1918] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:55:12.717693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097b487-5004-4ea8-8d59-680347544699 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:12.729466 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097b487-5004-4ea8-8d59-680347544699 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:12.734079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ac5ff-3db7-4763-8da1-e614713fd470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:12.735494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ac5ff-3db7-4763-8da1-e614713fd470 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:12.735910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ac5ff-3db7-4763-8da1-e614713fd470 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:12.736260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ac5ff-3db7-4763-8da1-e614713fd470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:12.738341 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f08ac5ff-3db7-4763-8da1-e614713fd470 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:12.748467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc583a1-f558-4764-a1e2-cba9c432e02b service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:12.750713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bdc583a1-f558-4764-a1e2-cba9c432e02b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:12.770132 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc583a1-f558-4764-a1e2-cba9c432e02b service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:12.776046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:12.776964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:12.777280 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:12.777504 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] PATH_INFO: `/v3/projects/52d379671ea64a5fbfe4817dfac4006c` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:12.778709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d379671ea64a5fbfe4817dfac4006c)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:12.780840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:12.810359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ac5ff-3db7-4763-8da1-e614713fd470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:12.811810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ac5ff-3db7-4763-8da1-e614713fd470 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:12.822574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 964/1919] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:12.843409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d379671ea64a5fbfe4817dfac4006c', 'request_id': 'req-f75968c1-bd00-401d-8834-ee6dd1a3f110'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:12.855966 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d379671ea64a5fbfe4817dfac4006c', 'request_id': 'req-f75968c1-bd00-401d-8834-ee6dd1a3f110'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:12.867248 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d379671ea64a5fbfe4817dfac4006c', 'request_id': 'req-f75968c1-bd00-401d-8834-ee6dd1a3f110'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:12.901008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] The token cache is being invalidate because project 52d379671ea64a5fbfe4817dfac4006c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:12.901713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:12.902688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:12.905427 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 956/1920] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:12 2022] DELETE /identity/v3/projects/52d379671ea64a5fbfe4817dfac4006c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:12.909603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ac5ff-3db7-4763-8da1-e614713fd470 service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:12.923666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ac5ff-3db7-4763-8da1-e614713fd470 service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:13.006156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:13.019185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b7587-e361-4b0f-95d5-e93daa3e4d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:13.021215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b7587-e361-4b0f-95d5-e93daa3e4d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:13.022114 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b7587-e361-4b0f-95d5-e93daa3e4d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:13.022813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b7587-e361-4b0f-95d5-e93daa3e4d71 admin admin] PATH_INFO: `/v3/users/da262dff9ad846dc81ae641256fef6ab` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:13.025400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b7587-e361-4b0f-95d5-e93daa3e4d71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da262dff9ad846dc81ae641256fef6ab)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:13.028481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b7587-e361-4b0f-95d5-e93daa3e4d71 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:13.093021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d10b7587-e361-4b0f-95d5-e93daa3e4d71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da262dff9ad846dc81ae641256fef6ab'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.102007 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d10b7587-e361-4b0f-95d5-e93daa3e4d71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da262dff9ad846dc81ae641256fef6ab'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.102870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d10b7587-e361-4b0f-95d5-e93daa3e4d71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da262dff9ad846dc81ae641256fef6ab'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.124083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d10b7587-e361-4b0f-95d5-e93daa3e4d71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da262dff9ad846dc81ae641256fef6ab'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.132420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d10b7587-e361-4b0f-95d5-e93daa3e4d71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da262dff9ad846dc81ae641256fef6ab'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.138787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 965/1921] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:12 2022] DELETE /identity/v3/users/da262dff9ad846dc81ae641256fef6ab => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:13.173039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:13.174568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:13.189366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75968c1-bd00-401d-8834-ee6dd1a3f110 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:13.242117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:13.264406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:13.264784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:13.265073 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:13.265387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] PATH_INFO: `/v3/users/f7cf632fe24048b59aa5aa59dddc879b` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:13.266888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7cf632fe24048b59aa5aa59dddc879b)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:13.268908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:13.327781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7cf632fe24048b59aa5aa59dddc879b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.335128 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7cf632fe24048b59aa5aa59dddc879b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.345199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7cf632fe24048b59aa5aa59dddc879b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.348575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7cf632fe24048b59aa5aa59dddc879b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.367431 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7cf632fe24048b59aa5aa59dddc879b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.380740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 957/1922] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:13 2022] DELETE /identity/v3/users/f7cf632fe24048b59aa5aa59dddc879b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:13.386355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4bc33-8363-4cd9-b505-4d59ad2a5d89 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:13.386770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4bc33-8363-4cd9-b505-4d59ad2a5d89 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:13.387052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4bc33-8363-4cd9-b505-4d59ad2a5d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:13.411057 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-efd4bc33-8363-4cd9-b505-4d59ad2a5d89 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:13.412511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-efd4bc33-8363-4cd9-b505-4d59ad2a5d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:13.486072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:13.488555 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:13.538230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 966/1923] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:13.549154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4bc33-8363-4cd9-b505-4d59ad2a5d89 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:13.551112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-efd4bc33-8363-4cd9-b505-4d59ad2a5d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:13.592828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f6c8b-5d01-439a-8eb4-39284df000e1 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:13.599381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:13.600294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:13.600680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:13.601072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] PATH_INFO: `/v3/projects/50efcf197176480989d8b3935799b1eb` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:13.602406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50efcf197176480989d8b3935799b1eb)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:13.604456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:13.680881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4bc33-8363-4cd9-b505-4d59ad2a5d89 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:13.687634 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50efcf197176480989d8b3935799b1eb', 'request_id': 'req-17b74559-a160-48d6-9691-b6d22fbae178'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.700379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50efcf197176480989d8b3935799b1eb', 'request_id': 'req-17b74559-a160-48d6-9691-b6d22fbae178'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.710376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50efcf197176480989d8b3935799b1eb', 'request_id': 'req-17b74559-a160-48d6-9691-b6d22fbae178'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.717712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:13.753105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] The token cache is being invalidate because project 50efcf197176480989d8b3935799b1eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:13.753402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.755823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:13.759861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:13.766649 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 958/1924] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:13 2022] DELETE /identity/v3/projects/50efcf197176480989d8b3935799b1eb => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:13.768453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:13.770909 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:13.811386 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aedf87-0ca9-4003-9619-f094089365cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:13.812558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aedf87-0ca9-4003-9619-f094089365cf service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:13.814649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aedf87-0ca9-4003-9619-f094089365cf service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:13.815348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aedf87-0ca9-4003-9619-f094089365cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:13.818606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-85aedf87-0ca9-4003-9619-f094089365cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:13.913294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b74559-a160-48d6-9691-b6d22fbae178 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:13.919103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:13.919999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:13.920299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:13.920663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] PATH_INFO: `/v3/users/b70d47088d2e4208a89696a7c970d9e9` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:13.922283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b70d47088d2e4208a89696a7c970d9e9)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:13.924694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:13.956462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aedf87-0ca9-4003-9619-f094089365cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:13.958179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aedf87-0ca9-4003-9619-f094089365cf service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:13.970868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 967/1925] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:55:14.039806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b70d47088d2e4208a89696a7c970d9e9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.056278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b70d47088d2e4208a89696a7c970d9e9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.073265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b70d47088d2e4208a89696a7c970d9e9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.074088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b70d47088d2e4208a89696a7c970d9e9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.108451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b70d47088d2e4208a89696a7c970d9e9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.126610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 959/1926] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:13 2022] DELETE /identity/v3/users/b70d47088d2e4208a89696a7c970d9e9 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:14.375522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aedf87-0ca9-4003-9619-f094089365cf service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:14.377664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-85aedf87-0ca9-4003-9619-f094089365cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:14.398090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aedf87-0ca9-4003-9619-f094089365cf service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:14.472699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:14.473607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:14.473899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:14.474180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] PATH_INFO: `/v3/projects/9ff2f24ea6cd44ca95506488ecf993af` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:14.475648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ff2f24ea6cd44ca95506488ecf993af)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:14.478017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:14.528154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ff2f24ea6cd44ca95506488ecf993af', 'request_id': 'req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.537615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ff2f24ea6cd44ca95506488ecf993af', 'request_id': 'req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.544425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ff2f24ea6cd44ca95506488ecf993af', 'request_id': 'req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.561491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] The token cache is being invalidate because project 9ff2f24ea6cd44ca95506488ecf993af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:14.561832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.562582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.564669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 968/1927] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:14 2022] DELETE /identity/v3/projects/9ff2f24ea6cd44ca95506488ecf993af => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:14.569409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:14.571256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:14.576626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:14.578764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:14.585630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578589bf-b310-434f-a7c6-e6eee414544c admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:14.600646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:14.654738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e5667-d5b4-4f73-afe1-c3a6b0a12ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:14.655640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e5667-d5b4-4f73-afe1-c3a6b0a12ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:14.655922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e5667-d5b4-4f73-afe1-c3a6b0a12ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:14.656188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e5667-d5b4-4f73-afe1-c3a6b0a12ec1 admin admin] PATH_INFO: `/v3/users/6f64c4f84dff4c3dbda2682f63bc55dc` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:14.658236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe9748-b149-4d2e-ae7d-5b5751f6fe4a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:14.659498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e5667-d5b4-4f73-afe1-c3a6b0a12ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f64c4f84dff4c3dbda2682f63bc55dc)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:14.661628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e5667-d5b4-4f73-afe1-c3a6b0a12ec1 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:14.664585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:14.666272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:14.666563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:14.666845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] PATH_INFO: `/v3/projects/d1c3c0ffb2184938a1a7f38897173305` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:14.668746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1c3c0ffb2184938a1a7f38897173305)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:14.671163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:14.681141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:14.735637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-3d2e5667-d5b4-4f73-afe1-c3a6b0a12ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f64c4f84dff4c3dbda2682f63bc55dc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.746899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1c3c0ffb2184938a1a7f38897173305', 'request_id': 'req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.747104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-3d2e5667-d5b4-4f73-afe1-c3a6b0a12ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f64c4f84dff4c3dbda2682f63bc55dc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.757015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-3d2e5667-d5b4-4f73-afe1-c3a6b0a12ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f64c4f84dff4c3dbda2682f63bc55dc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.758174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-3d2e5667-d5b4-4f73-afe1-c3a6b0a12ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f64c4f84dff4c3dbda2682f63bc55dc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.758353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1c3c0ffb2184938a1a7f38897173305', 'request_id': 'req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.765729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1c3c0ffb2184938a1a7f38897173305', 'request_id': 'req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.773626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-3d2e5667-d5b4-4f73-afe1-c3a6b0a12ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f64c4f84dff4c3dbda2682f63bc55dc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.777863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] The token cache is being invalidate because project d1c3c0ffb2184938a1a7f38897173305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:14.778316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.779079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7ddfe784-f74c-4f71-b5d2-f2fe6b40f691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:14.784159 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 960/1928] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:14 2022] DELETE /identity/v3/users/6f64c4f84dff4c3dbda2682f63bc55dc => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:14.784381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 969/1929] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:14 2022] DELETE /identity/v3/projects/d1c3c0ffb2184938a1a7f38897173305 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:14.790732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897633b2-ecba-45c9-b36a-976375cfb68b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:14.790875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897633b2-ecba-45c9-b36a-976375cfb68b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:14.791143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897633b2-ecba-45c9-b36a-976375cfb68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:14.810526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-897633b2-ecba-45c9-b36a-976375cfb68b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:14.811818 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-897633b2-ecba-45c9-b36a-976375cfb68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:14.883928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 961/1930] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:14.892069 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71cf76-858a-4a75-ab1d-cdd0fcc3bc25 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:14.892389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71cf76-858a-4a75-ab1d-cdd0fcc3bc25 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:14.892674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71cf76-858a-4a75-ab1d-cdd0fcc3bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:14.912838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-6c71cf76-858a-4a75-ab1d-cdd0fcc3bc25 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:14.914083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6c71cf76-858a-4a75-ab1d-cdd0fcc3bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:14.922972 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 970/1931] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:14.927074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897633b2-ecba-45c9-b36a-976375cfb68b None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:14.937297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897633b2-ecba-45c9-b36a-976375cfb68b None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:14.943287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fcbdd8-b053-4b73-88ff-888b749b8c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:14.944022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcbdd8-b053-4b73-88ff-888b749b8c9d admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:14.944312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcbdd8-b053-4b73-88ff-888b749b8c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:14.944620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcbdd8-b053-4b73-88ff-888b749b8c9d admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:14.945392 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fcbdd8-b053-4b73-88ff-888b749b8c9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:14.945695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fcbdd8-b053-4b73-88ff-888b749b8c9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:14.947358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fcbdd8-b053-4b73-88ff-888b749b8c9d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:14.961973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 962/1932] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:55:14.966348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71cf76-858a-4a75-ab1d-cdd0fcc3bc25 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:14.975815 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71cf76-858a-4a75-ab1d-cdd0fcc3bc25 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:14.980708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bb67d-8479-45db-b487-ce7e4487122a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:14.981371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bb67d-8479-45db-b487-ce7e4487122a admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:14.981665 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bb67d-8479-45db-b487-ce7e4487122a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:14.981933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bb67d-8479-45db-b487-ce7e4487122a admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:14.982632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bb67d-8479-45db-b487-ce7e4487122a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:14.984585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bb67d-8479-45db-b487-ce7e4487122a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.012732 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 971/1933] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:14 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:15.017659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fcbdd8-b053-4b73-88ff-888b749b8c9d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.029795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fcbdd8-b053-4b73-88ff-888b749b8c9d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.095743 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bb67d-8479-45db-b487-ce7e4487122a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.097876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-259bb67d-8479-45db-b487-ce7e4487122a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:15.119295 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec068eab-e814-4b7e-951a-3841cb60b4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.120212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec068eab-e814-4b7e-951a-3841cb60b4bb admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.120611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec068eab-e814-4b7e-951a-3841cb60b4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.121053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec068eab-e814-4b7e-951a-3841cb60b4bb admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.121773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec068eab-e814-4b7e-951a-3841cb60b4bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.123769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec068eab-e814-4b7e-951a-3841cb60b4bb admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.152383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 963/1934] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:15 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:15.156120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec068eab-e814-4b7e-951a-3841cb60b4bb admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.167562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec068eab-e814-4b7e-951a-3841cb60b4bb admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.172888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f57a4e-b870-4473-9064-acc877ea4600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.173648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57a4e-b870-4473-9064-acc877ea4600 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.173961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57a4e-b870-4473-9064-acc877ea4600 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.174245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f57a4e-b870-4473-9064-acc877ea4600 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.175109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f57a4e-b870-4473-9064-acc877ea4600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.176779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f57a4e-b870-4473-9064-acc877ea4600 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.179289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:15.192791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 964/1935] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:55:15.195243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bb67d-8479-45db-b487-ce7e4487122a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.195651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f57a4e-b870-4473-9064-acc877ea4600 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.200356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.201295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.201586 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.201867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] PATH_INFO: `/v3/projects/6cbb402bfa134a708699d0301c794fc4` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.203102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cbb402bfa134a708699d0301c794fc4)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.205464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.218698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f57a4e-b870-4473-9064-acc877ea4600 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.225264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2053e-83e5-40c8-b7a7-1b446f1c5d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.226427 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2053e-83e5-40c8-b7a7-1b446f1c5d9c admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.226991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2053e-83e5-40c8-b7a7-1b446f1c5d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.227279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2053e-83e5-40c8-b7a7-1b446f1c5d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.228296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2053e-83e5-40c8-b7a7-1b446f1c5d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.229991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb2053e-83e5-40c8-b7a7-1b446f1c5d9c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.254919 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 965/1936] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:55:15.261326 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:15.266619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2053e-83e5-40c8-b7a7-1b446f1c5d9c admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.281301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2053e-83e5-40c8-b7a7-1b446f1c5d9c admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.287664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891402b-7146-4edf-a21d-81e5fd64ddb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.288625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891402b-7146-4edf-a21d-81e5fd64ddb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.288940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891402b-7146-4edf-a21d-81e5fd64ddb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.289219 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891402b-7146-4edf-a21d-81e5fd64ddb7 admin admin] PATH_INFO: `/v3/projects/72c814e2b46244db93e801a78763d880/users/483b27bfe5674a528c29a15a054cc617/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.310108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cbb402bfa134a708699d0301c794fc4', 'request_id': 'req-424293b5-4d2b-4079-bfec-1c1e8000a6a2'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:15.319769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891402b-7146-4edf-a21d-81e5fd64ddb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72c814e2b46244db93e801a78763d880, user_id=483b27bfe5674a528c29a15a054cc617, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.320048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cbb402bfa134a708699d0301c794fc4', 'request_id': 'req-424293b5-4d2b-4079-bfec-1c1e8000a6a2'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:15.323523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891402b-7146-4edf-a21d-81e5fd64ddb7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.327259 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cbb402bfa134a708699d0301c794fc4', 'request_id': 'req-424293b5-4d2b-4079-bfec-1c1e8000a6a2'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:15.341215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 966/1937] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:15 2022] PUT /identity/v3/projects/72c814e2b46244db93e801a78763d880/users/483b27bfe5674a528c29a15a054cc617/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:15.343077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] The token cache is being invalidate because project 6cbb402bfa134a708699d0301c794fc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:15.343374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:15.345182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-424293b5-4d2b-4079-bfec-1c1e8000a6a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:15.346991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 972/1938] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:15 2022] DELETE /identity/v3/projects/6cbb402bfa134a708699d0301c794fc4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:15.347115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8456b-f34a-4825-98d2-400efc830f0c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.350588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8456b-f34a-4825-98d2-400efc830f0c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.351132 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8456b-f34a-4825-98d2-400efc830f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.351795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489537-06ef-459e-b4fa-bf1c30ddf928 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.351957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489537-06ef-459e-b4fa-bf1c30ddf928 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.352746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489537-06ef-459e-b4fa-bf1c30ddf928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.371314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3ef8456b-f34a-4825-98d2-400efc830f0c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:15.372707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3ef8456b-f34a-4825-98d2-400efc830f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:15.377339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-50489537-06ef-459e-b4fa-bf1c30ddf928 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:15.378820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-50489537-06ef-459e-b4fa-bf1c30ddf928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:15.458685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 967/1939] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:15.470780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8456b-f34a-4825-98d2-400efc830f0c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.472642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3ef8456b-f34a-4825-98d2-400efc830f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:15.476064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 973/1940] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:15.485691 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882cb5d-4c3c-417e-bc4e-418d89e0e74a None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.485691 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882cb5d-4c3c-417e-bc4e-418d89e0e74a None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.485691 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882cb5d-4c3c-417e-bc4e-418d89e0e74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.504921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-1882cb5d-4c3c-417e-bc4e-418d89e0e74a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:15.506167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1882cb5d-4c3c-417e-bc4e-418d89e0e74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:15.515882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 974/1941] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:15.521060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882cb5d-4c3c-417e-bc4e-418d89e0e74a None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.533456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882cb5d-4c3c-417e-bc4e-418d89e0e74a None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.537739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8456b-f34a-4825-98d2-400efc830f0c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.541745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8bba4-7ffd-462c-a090-97e5ba571090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.542456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8bba4-7ffd-462c-a090-97e5ba571090 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.542695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8bba4-7ffd-462c-a090-97e5ba571090 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.543009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8bba4-7ffd-462c-a090-97e5ba571090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.549487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3b2a5-b4e2-418e-8bb4-8cb7c8b294d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.550181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3b2a5-b4e2-418e-8bb4-8cb7c8b294d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.550479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3b2a5-b4e2-418e-8bb4-8cb7c8b294d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.550827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3b2a5-b4e2-418e-8bb4-8cb7c8b294d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.551613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3b2a5-b4e2-418e-8bb4-8cb7c8b294d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:15.551929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3b2a5-b4e2-418e-8bb4-8cb7c8b294d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.553613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3b2a5-b4e2-418e-8bb4-8cb7c8b294d2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.555441 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8bba4-7ffd-462c-a090-97e5ba571090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.556967 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8bba4-7ffd-462c-a090-97e5ba571090 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.570385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 975/1942] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:55:15.570579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 968/1943] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:55:15.571500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3b2a5-b4e2-418e-8bb4-8cb7c8b294d2 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.583889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3b2a5-b4e2-418e-8bb4-8cb7c8b294d2 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.589270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81512ac4-0954-462f-b0e9-0a169f2fac35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.598666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81512ac4-0954-462f-b0e9-0a169f2fac35 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.598666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81512ac4-0954-462f-b0e9-0a169f2fac35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.598666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81512ac4-0954-462f-b0e9-0a169f2fac35 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.598666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81512ac4-0954-462f-b0e9-0a169f2fac35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.598666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81512ac4-0954-462f-b0e9-0a169f2fac35 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.626381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 976/1944] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:15 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:15.627749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8bba4-7ffd-462c-a090-97e5ba571090 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.638171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8bba4-7ffd-462c-a090-97e5ba571090 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.704477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e09b5-5e61-41a7-a932-f9bcabd1d87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.705156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e09b5-5e61-41a7-a932-f9bcabd1d87d admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.705470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e09b5-5e61-41a7-a932-f9bcabd1d87d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.705744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e09b5-5e61-41a7-a932-f9bcabd1d87d admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.706443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e09b5-5e61-41a7-a932-f9bcabd1d87d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.708408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e09b5-5e61-41a7-a932-f9bcabd1d87d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.719967 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:15.738325 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 969/1945] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:15 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:55:15.739900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81512ac4-0954-462f-b0e9-0a169f2fac35 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.751901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81512ac4-0954-462f-b0e9-0a169f2fac35 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.757147 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027415ea-26d0-4ca9-bb85-4b1aee80a780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.757831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027415ea-26d0-4ca9-bb85-4b1aee80a780 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.758119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027415ea-26d0-4ca9-bb85-4b1aee80a780 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.758403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027415ea-26d0-4ca9-bb85-4b1aee80a780 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.759139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027415ea-26d0-4ca9-bb85-4b1aee80a780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.760808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027415ea-26d0-4ca9-bb85-4b1aee80a780 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.774923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 977/1946] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:15.776962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e09b5-5e61-41a7-a932-f9bcabd1d87d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.789191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e09b5-5e61-41a7-a932-f9bcabd1d87d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.795244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f9df2-18f1-4613-a7c3-3dd482a8b81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.795957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f9df2-18f1-4613-a7c3-3dd482a8b81a admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.796244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f9df2-18f1-4613-a7c3-3dd482a8b81a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.796627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f9df2-18f1-4613-a7c3-3dd482a8b81a admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.797371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f9df2-18f1-4613-a7c3-3dd482a8b81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.799001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f9df2-18f1-4613-a7c3-3dd482a8b81a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.809593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 970/1947] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:15.814155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027415ea-26d0-4ca9-bb85-4b1aee80a780 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:15.825597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027415ea-26d0-4ca9-bb85-4b1aee80a780 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:15.831037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdbca5-a9a5-4a61-924b-50da04deb82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:15.831987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdbca5-a9a5-4a61-924b-50da04deb82e admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.832252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdbca5-a9a5-4a61-924b-50da04deb82e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.832576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdbca5-a9a5-4a61-924b-50da04deb82e admin admin] PATH_INFO: `/v3/projects/69986b31fd064e4f824856578e732a39/users/b621ebc1564a4719a0b8aa147af2194f/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.848923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdbca5-a9a5-4a61-924b-50da04deb82e admin admin] RBAC: Authorizing `identity:create_grant(project_id=69986b31fd064e4f824856578e732a39, user_id=b621ebc1564a4719a0b8aa147af2194f, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:15.851660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdbca5-a9a5-4a61-924b-50da04deb82e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:15.872505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 978/1948] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:15 2022] PUT /identity/v3/projects/69986b31fd064e4f824856578e732a39/users/b621ebc1564a4719a0b8aa147af2194f/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:15.877603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a6bb0-e2ea-4dab-a5b7-705ebbaa2774 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:15.877905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a6bb0-e2ea-4dab-a5b7-705ebbaa2774 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:15.878222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a6bb0-e2ea-4dab-a5b7-705ebbaa2774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:15.907792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-352a6bb0-e2ea-4dab-a5b7-705ebbaa2774 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:15.909897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-352a6bb0-e2ea-4dab-a5b7-705ebbaa2774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:16.001429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 971/1949] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:16.008207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdbca5-a9a5-4a61-924b-50da04deb82e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:16.044883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:16.082551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdbca5-a9a5-4a61-924b-50da04deb82e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:16.086894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7cf4cc-1535-4fe7-963c-1c53dae45de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:16.087579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cf4cc-1535-4fe7-963c-1c53dae45de9 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:16.087868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cf4cc-1535-4fe7-963c-1c53dae45de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:16.088153 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7cf4cc-1535-4fe7-963c-1c53dae45de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:16.100294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7cf4cc-1535-4fe7-963c-1c53dae45de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:16.101930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7cf4cc-1535-4fe7-963c-1c53dae45de9 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:16.113045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 979/1950] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:16.702057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313b5a9-2f4b-46fb-9ba8-06e0765b69d3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:16.702429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313b5a9-2f4b-46fb-9ba8-06e0765b69d3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:16.702609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313b5a9-2f4b-46fb-9ba8-06e0765b69d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:16.760066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-1313b5a9-2f4b-46fb-9ba8-06e0765b69d3 None None] MFA Rules not processed for user `483b27bfe5674a528c29a15a054cc617`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:16.762041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1313b5a9-2f4b-46fb-9ba8-06e0765b69d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:16.917086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 972/1951] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:16.930508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7cf4cc-1535-4fe7-963c-1c53dae45de9 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:16.932306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8c7cf4cc-1535-4fe7-963c-1c53dae45de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:17.054044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7cf4cc-1535-4fe7-963c-1c53dae45de9 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:17.058378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dbad4-4927-4c38-98d3-c99641c4155d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:17.072378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dbad4-4927-4c38-98d3-c99641c4155d service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:17.072776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dbad4-4927-4c38-98d3-c99641c4155d service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:17.073283 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dbad4-4927-4c38-98d3-c99641c4155d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:17.084212 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188dbad4-4927-4c38-98d3-c99641c4155d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:17.086932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188dbad4-4927-4c38-98d3-c99641c4155d service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:17.099168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 980/1952] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:17.308110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e515860-2b50-4c26-9c51-42b77f5f25c0 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:17.308651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e515860-2b50-4c26-9c51-42b77f5f25c0 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:17.309083 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e515860-2b50-4c26-9c51-42b77f5f25c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:17.371869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-0e515860-2b50-4c26-9c51-42b77f5f25c0 None None] MFA Rules not processed for user `b621ebc1564a4719a0b8aa147af2194f`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:17.373477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0e515860-2b50-4c26-9c51-42b77f5f25c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:17.501637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 973/1953] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:17.506767 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dbad4-4927-4c38-98d3-c99641c4155d service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:17.522686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dbad4-4927-4c38-98d3-c99641c4155d service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:17.546077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b439da-43a4-4534-8750-66bd2047b909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:17.549364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b439da-43a4-4534-8750-66bd2047b909 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:17.549880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b439da-43a4-4534-8750-66bd2047b909 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:17.550308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b439da-43a4-4534-8750-66bd2047b909 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:17.553654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b439da-43a4-4534-8750-66bd2047b909 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:17.555776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b439da-43a4-4534-8750-66bd2047b909 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:17.604368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 981/1954] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:17 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:17.607917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e515860-2b50-4c26-9c51-42b77f5f25c0 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:17.615510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:17.620282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e515860-2b50-4c26-9c51-42b77f5f25c0 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:17.682596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dc060-aadc-480f-b5f6-28094eb6fd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:17.683374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dc060-aadc-480f-b5f6-28094eb6fd97 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:17.683659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dc060-aadc-480f-b5f6-28094eb6fd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:17.683938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dc060-aadc-480f-b5f6-28094eb6fd97 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:17.685655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dc060-aadc-480f-b5f6-28094eb6fd97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:17.687008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dc060-aadc-480f-b5f6-28094eb6fd97 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:17.715829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 974/1955] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:17 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:17.720670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b439da-43a4-4534-8750-66bd2047b909 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:17.731578 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b439da-43a4-4534-8750-66bd2047b909 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:17.736481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ae20f-550c-4691-a3e1-acc227bed3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:17.737170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ae20f-550c-4691-a3e1-acc227bed3fe admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:17.737446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ae20f-550c-4691-a3e1-acc227bed3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:17.737723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ae20f-550c-4691-a3e1-acc227bed3fe admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:17.738457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ae20f-550c-4691-a3e1-acc227bed3fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:17.740046 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ae20f-550c-4691-a3e1-acc227bed3fe admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:17.752453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 982/1956] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:17.757956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dc060-aadc-480f-b5f6-28094eb6fd97 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:17.768560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dc060-aadc-480f-b5f6-28094eb6fd97 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:17.774823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306f1ef-e76c-4ae9-b160-3600316c82b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:17.776153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f1ef-e76c-4ae9-b160-3600316c82b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:17.776536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f1ef-e76c-4ae9-b160-3600316c82b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:17.776898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f1ef-e76c-4ae9-b160-3600316c82b5 admin admin] PATH_INFO: `/v3/projects/de628dd84ca743d98bbae8b13fe8d4c4/users/370c788de1cc481591137176bb4ed486/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:17.793313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306f1ef-e76c-4ae9-b160-3600316c82b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de628dd84ca743d98bbae8b13fe8d4c4, user_id=370c788de1cc481591137176bb4ed486, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:17.796190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306f1ef-e76c-4ae9-b160-3600316c82b5 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:17.815004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 975/1957] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:17 2022] PUT /identity/v3/projects/de628dd84ca743d98bbae8b13fe8d4c4/users/370c788de1cc481591137176bb4ed486/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:17.819127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ae20f-550c-4691-a3e1-acc227bed3fe admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:17.831909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ae20f-550c-4691-a3e1-acc227bed3fe admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:17.904199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293687ba-20ed-43d0-9a97-e075ee7689fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:17.905026 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293687ba-20ed-43d0-9a97-e075ee7689fd admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:17.905404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293687ba-20ed-43d0-9a97-e075ee7689fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:17.905720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293687ba-20ed-43d0-9a97-e075ee7689fd admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:17.906603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293687ba-20ed-43d0-9a97-e075ee7689fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:17.908639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293687ba-20ed-43d0-9a97-e075ee7689fd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:17.920659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 983/1958] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:17 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:17.928894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306f1ef-e76c-4ae9-b160-3600316c82b5 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:17.941191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306f1ef-e76c-4ae9-b160-3600316c82b5 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:17.945998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e07b1d-a526-42c1-be57-b344e7a5a0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:17.946895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e07b1d-a526-42c1-be57-b344e7a5a0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:17.947179 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e07b1d-a526-42c1-be57-b344e7a5a0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:17.947457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e07b1d-a526-42c1-be57-b344e7a5a0d7 admin admin] PATH_INFO: `/v3/projects/de628dd84ca743d98bbae8b13fe8d4c4/users/370c788de1cc481591137176bb4ed486/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:17.949612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e07b1d-a526-42c1-be57-b344e7a5a0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de628dd84ca743d98bbae8b13fe8d4c4, user_id=370c788de1cc481591137176bb4ed486, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:17.952306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e07b1d-a526-42c1-be57-b344e7a5a0d7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:17.964290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 976/1959] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:17 2022] PUT /identity/v3/projects/de628dd84ca743d98bbae8b13fe8d4c4/users/370c788de1cc481591137176bb4ed486/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:18.653362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df9584-e9b1-4e13-b315-abb2b9487246 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:18.653944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df9584-e9b1-4e13-b315-abb2b9487246 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:18.653944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df9584-e9b1-4e13-b315-abb2b9487246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:18.692839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-61df9584-e9b1-4e13-b315-abb2b9487246 None None] MFA Rules not processed for user `370c788de1cc481591137176bb4ed486`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:18.694084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-61df9584-e9b1-4e13-b315-abb2b9487246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:18.722078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:18.775450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 984/1960] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:18.787960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e07b1d-a526-42c1-be57-b344e7a5a0d7 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:18.847459 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e07b1d-a526-42c1-be57-b344e7a5a0d7 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:18.851241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33f16b-aa58-47eb-ae89-08eb74ae53b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:18.852046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f16b-aa58-47eb-ae89-08eb74ae53b6 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:18.852358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f16b-aa58-47eb-ae89-08eb74ae53b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:18.852646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f16b-aa58-47eb-ae89-08eb74ae53b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:18.916984 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33f16b-aa58-47eb-ae89-08eb74ae53b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:18.918428 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33f16b-aa58-47eb-ae89-08eb74ae53b6 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:18.928189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 977/1961] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:23.606000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabca2c-3bda-478d-b796-94e3e97aefee None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:23.606521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabca2c-3bda-478d-b796-94e3e97aefee None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:23.606599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabca2c-3bda-478d-b796-94e3e97aefee None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:23.607968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 985/1962] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 20:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:55:23.618561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8dfcd-a13a-4867-878b-8ccca6845463 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:23.619498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8dfcd-a13a-4867-878b-8ccca6845463 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:23.619807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8dfcd-a13a-4867-878b-8ccca6845463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:23.641258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-7fb8dfcd-a13a-4867-878b-8ccca6845463 None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:23.642510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7fb8dfcd-a13a-4867-878b-8ccca6845463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:23.651158 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:23.729511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 978/1963] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 20:55:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:23.742083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabca2c-3bda-478d-b796-94e3e97aefee None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:23.744369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fdabca2c-3bda-478d-b796-94e3e97aefee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:23.764898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabca2c-3bda-478d-b796-94e3e97aefee None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:23.771291 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddc13c-6c08-4adc-854d-743bb094c70c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:23.772138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddc13c-6c08-4adc-854d-743bb094c70c service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:23.772310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddc13c-6c08-4adc-854d-743bb094c70c service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:23.772693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddc13c-6c08-4adc-854d-743bb094c70c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:23.789463 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ddc13c-6c08-4adc-854d-743bb094c70c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:23.789463 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ddc13c-6c08-4adc-854d-743bb094c70c service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:23.810883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 986/1964] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:23.872549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521ba63-4e58-43c4-bca6-ee034d8336b5 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:23.872822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521ba63-4e58-43c4-bca6-ee034d8336b5 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:23.873106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521ba63-4e58-43c4-bca6-ee034d8336b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:23.892822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-6521ba63-4e58-43c4-bca6-ee034d8336b5 None None] MFA Rules not processed for user `483b27bfe5674a528c29a15a054cc617`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:23.894056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6521ba63-4e58-43c4-bca6-ee034d8336b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:23.954973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 979/1965] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:24.594609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddc13c-6c08-4adc-854d-743bb094c70c service nova] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:24.596395 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-85ddc13c-6c08-4adc-854d-743bb094c70c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:24.608376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddc13c-6c08-4adc-854d-743bb094c70c service nova] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:24.613042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828212b-e3da-4f66-9492-6835c9f45820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:24.613690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828212b-e3da-4f66-9492-6835c9f45820 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:24.613974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828212b-e3da-4f66-9492-6835c9f45820 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:24.614253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828212b-e3da-4f66-9492-6835c9f45820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:24.624187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828212b-e3da-4f66-9492-6835c9f45820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:24.625984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828212b-e3da-4f66-9492-6835c9f45820 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:24.637833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 987/1966] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:30.413172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521ba63-4e58-43c4-bca6-ee034d8336b5 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:30.414632 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6521ba63-4e58-43c4-bca6-ee034d8336b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:30.526431 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521ba63-4e58-43c4-bca6-ee034d8336b5 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:30.532247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:30.533162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:30.533444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:30.533722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] PATH_INFO: `/v3/users/5ca36333601646fdbd295c6cecf72882` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:30.535603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca36333601646fdbd295c6cecf72882)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:30.539150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:30.631794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca36333601646fdbd295c6cecf72882'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:30.635695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:30.643991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca36333601646fdbd295c6cecf72882'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:30.660449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca36333601646fdbd295c6cecf72882'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:30.661215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca36333601646fdbd295c6cecf72882'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:30.679449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca36333601646fdbd295c6cecf72882'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:30.687881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 980/1967] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:30 2022] DELETE /identity/v3/users/5ca36333601646fdbd295c6cecf72882 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:30.696131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6a8fa-965d-4361-a9cb-0e75769dd8d4 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:30.697163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6a8fa-965d-4361-a9cb-0e75769dd8d4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:30.697163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6a8fa-965d-4361-a9cb-0e75769dd8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:30.728365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-56f6a8fa-965d-4361-a9cb-0e75769dd8d4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:30.730277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-56f6a8fa-965d-4361-a9cb-0e75769dd8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:30.819811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:30.822166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:30.844857 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 988/1968] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:30.859943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6a8fa-965d-4361-a9cb-0e75769dd8d4 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:30.861458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-56f6a8fa-965d-4361-a9cb-0e75769dd8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:30.935231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf439ee-ee14-423a-8eca-71b8fa6a71d0 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:30.941834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:30.942839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:30.943244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:30.943619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] PATH_INFO: `/v3/users/850e86e60d704092a1bb5a96db587196` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:30.945339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=850e86e60d704092a1bb5a96db587196)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:30.949124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:30.970758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6a8fa-965d-4361-a9cb-0e75769dd8d4 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:30.975871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359f5c0-5148-4ce7-a447-1cc1130007e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:30.977221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f5c0-5148-4ce7-a447-1cc1130007e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:30.977648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f5c0-5148-4ce7-a447-1cc1130007e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:30.978024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f5c0-5148-4ce7-a447-1cc1130007e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:31.005851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359f5c0-5148-4ce7-a447-1cc1130007e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:31.010338 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359f5c0-5148-4ce7-a447-1cc1130007e6 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:31.042845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 989/1969] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:55:31.056302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '850e86e60d704092a1bb5a96db587196'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:31.084415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '850e86e60d704092a1bb5a96db587196'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:31.096605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '850e86e60d704092a1bb5a96db587196'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:31.097510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '850e86e60d704092a1bb5a96db587196'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:31.120294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '850e86e60d704092a1bb5a96db587196'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:31.146125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 981/1970] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:30 2022] DELETE /identity/v3/users/850e86e60d704092a1bb5a96db587196 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:31.148919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484db99-9f8f-45ec-a5e9-d890e66619d8 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:31.148919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484db99-9f8f-45ec-a5e9-d890e66619d8 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:31.149255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484db99-9f8f-45ec-a5e9-d890e66619d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:31.173937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-6484db99-9f8f-45ec-a5e9-d890e66619d8 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:31.175244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6484db99-9f8f-45ec-a5e9-d890e66619d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:31.259663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:31.295804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 990/1971] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:31.323816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:31.326469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:31.507285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:31.519986 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d22af-5a7e-495e-9386-9c7fc6bbc0a8 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:31.526870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ce3cf-d226-4ed3-a237-344f6d77cb91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:31.527582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ce3cf-d226-4ed3-a237-344f6d77cb91 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:31.528179 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ce3cf-d226-4ed3-a237-344f6d77cb91 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:31.528628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ce3cf-d226-4ed3-a237-344f6d77cb91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:31.541344 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ce3cf-d226-4ed3-a237-344f6d77cb91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:31.543355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ce3cf-d226-4ed3-a237-344f6d77cb91 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:31.561413 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 982/1972] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:31.618814 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484db99-9f8f-45ec-a5e9-d890e66619d8 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:31.622975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6484db99-9f8f-45ec-a5e9-d890e66619d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:31.652294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484db99-9f8f-45ec-a5e9-d890e66619d8 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:31.665548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:31.666183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:31.666465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:31.666741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] PATH_INFO: `/v3/projects/cdafd6a2f8d5417690241ae3d8be1437` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:31.668082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdafd6a2f8d5417690241ae3d8be1437)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:31.670037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:31.750913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdafd6a2f8d5417690241ae3d8be1437', 'request_id': 'req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:31.762142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdafd6a2f8d5417690241ae3d8be1437', 'request_id': 'req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:31.771450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdafd6a2f8d5417690241ae3d8be1437', 'request_id': 'req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:31.792290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] The token cache is being invalidate because project cdafd6a2f8d5417690241ae3d8be1437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:31.793578 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:31.794502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:31.796486 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 991/1973] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:31 2022] DELETE /identity/v3/projects/cdafd6a2f8d5417690241ae3d8be1437 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:31.801466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ce3cf-d226-4ed3-a237-344f6d77cb91 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:31.816170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ce3cf-d226-4ed3-a237-344f6d77cb91 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:31.931357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:31.932779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:31.933272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:31.933773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] PATH_INFO: `/v3/users/e0cf74dabfab45428ec0d4c56298d360` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:31.936296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0cf74dabfab45428ec0d4c56298d360)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:31.939330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:32.019009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0cf74dabfab45428ec0d4c56298d360'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.033125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0cf74dabfab45428ec0d4c56298d360'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.056851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0cf74dabfab45428ec0d4c56298d360'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.058025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0cf74dabfab45428ec0d4c56298d360'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.095796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0cf74dabfab45428ec0d4c56298d360'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.119405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 983/1974] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:31 2022] DELETE /identity/v3/users/e0cf74dabfab45428ec0d4c56298d360 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:32.393764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:32.395529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:32.410301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1cbce-1991-4abc-a5c1-3a67d03d83a5 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:32.514107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:32.515043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:32.515513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:32.515799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] PATH_INFO: `/v3/projects/092d039a2b1c4b70819b1f3aa968ec92` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:32.518001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=092d039a2b1c4b70819b1f3aa968ec92)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:32.520804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:32.626492 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '092d039a2b1c4b70819b1f3aa968ec92', 'request_id': 'req-7aabdc44-133d-48a4-a86c-74f898a8c8fc'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.646368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '092d039a2b1c4b70819b1f3aa968ec92', 'request_id': 'req-7aabdc44-133d-48a4-a86c-74f898a8c8fc'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.662317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '092d039a2b1c4b70819b1f3aa968ec92', 'request_id': 'req-7aabdc44-133d-48a4-a86c-74f898a8c8fc'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.667485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:32.680713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] The token cache is being invalidate because project 092d039a2b1c4b70819b1f3aa968ec92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:32.681449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.683587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.697393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 992/1975] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:32 2022] DELETE /identity/v3/projects/092d039a2b1c4b70819b1f3aa968ec92 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:32.698123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:32.699591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:32.722752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975184b-67d1-46fa-932c-2dd1be364edd admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:32.827414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978973b1-d4a1-49e6-a96d-564128dbf629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:32.828857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978973b1-d4a1-49e6-a96d-564128dbf629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:32.829388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978973b1-d4a1-49e6-a96d-564128dbf629 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:32.829912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978973b1-d4a1-49e6-a96d-564128dbf629 admin admin] PATH_INFO: `/v3/users/7552108b01a84a379f96931f6585c5ad` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:32.832557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978973b1-d4a1-49e6-a96d-564128dbf629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7552108b01a84a379f96931f6585c5ad)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:32.836581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978973b1-d4a1-49e6-a96d-564128dbf629 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:32.840137 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:32.842711 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:32.857969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:32.871315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabdc44-133d-48a4-a86c-74f898a8c8fc admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:32.878407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:32.879340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:32.879786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:32.880522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] PATH_INFO: `/v3/projects/7a8a5d06fbdd46f3adb29440b48ffaaa` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:32.883822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a8a5d06fbdd46f3adb29440b48ffaaa)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:32.886606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:32.941550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-978973b1-d4a1-49e6-a96d-564128dbf629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7552108b01a84a379f96931f6585c5ad'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.955022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-978973b1-d4a1-49e6-a96d-564128dbf629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7552108b01a84a379f96931f6585c5ad'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.963658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a8a5d06fbdd46f3adb29440b48ffaaa', 'request_id': 'req-ea22bfe0-dc4b-4873-ba6c-564350d86204'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.966214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-978973b1-d4a1-49e6-a96d-564128dbf629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7552108b01a84a379f96931f6585c5ad'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.966975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-978973b1-d4a1-49e6-a96d-564128dbf629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7552108b01a84a379f96931f6585c5ad'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.975976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a8a5d06fbdd46f3adb29440b48ffaaa', 'request_id': 'req-ea22bfe0-dc4b-4873-ba6c-564350d86204'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.987086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a8a5d06fbdd46f3adb29440b48ffaaa', 'request_id': 'req-ea22bfe0-dc4b-4873-ba6c-564350d86204'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:32.991011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-978973b1-d4a1-49e6-a96d-564128dbf629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7552108b01a84a379f96931f6585c5ad'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:33.005186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 984/1976] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:32 2022] DELETE /identity/v3/users/7552108b01a84a379f96931f6585c5ad => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:33.006797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] The token cache is being invalidate because project 7a8a5d06fbdd46f3adb29440b48ffaaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:33.007169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:33.008311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ea22bfe0-dc4b-4873-ba6c-564350d86204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:33.010616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 993/1977] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:32 2022] DELETE /identity/v3/projects/7a8a5d06fbdd46f3adb29440b48ffaaa => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:33.016258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127f688-72e6-44a1-a321-b6b33318e1db None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:33.016640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127f688-72e6-44a1-a321-b6b33318e1db None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:33.016937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127f688-72e6-44a1-a321-b6b33318e1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:33.047457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-0127f688-72e6-44a1-a321-b6b33318e1db None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:33.049124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0127f688-72e6-44a1-a321-b6b33318e1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:33.160168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda8926-f77c-41e0-ad0d-839651c53d02 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:33.160506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 985/1978] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:33.160947 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda8926-f77c-41e0-ad0d-839651c53d02 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:33.161238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda8926-f77c-41e0-ad0d-839651c53d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:33.205167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-4bda8926-f77c-41e0-ad0d-839651c53d02 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:33.205167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4bda8926-f77c-41e0-ad0d-839651c53d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:33.221999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 994/1979] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:33.227900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127f688-72e6-44a1-a321-b6b33318e1db None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:33.262894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127f688-72e6-44a1-a321-b6b33318e1db None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:33.272108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab52626-88fe-491e-a183-6ea6a79b1ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:33.273463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab52626-88fe-491e-a183-6ea6a79b1ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:33.273824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab52626-88fe-491e-a183-6ea6a79b1ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:33.274324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab52626-88fe-491e-a183-6ea6a79b1ff3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:33.275425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab52626-88fe-491e-a183-6ea6a79b1ff3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:33.275819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab52626-88fe-491e-a183-6ea6a79b1ff3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:33.278152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab52626-88fe-491e-a183-6ea6a79b1ff3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:33.296832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 986/1980] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:55:33.300819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda8926-f77c-41e0-ad0d-839651c53d02 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:33.313290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda8926-f77c-41e0-ad0d-839651c53d02 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:33.319154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9fe85-176f-4ba9-800d-759589cec01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:33.319830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9fe85-176f-4ba9-800d-759589cec01a admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:33.320119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9fe85-176f-4ba9-800d-759589cec01a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:33.320432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9fe85-176f-4ba9-800d-759589cec01a admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:33.321153 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9fe85-176f-4ba9-800d-759589cec01a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:33.324881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9fe85-176f-4ba9-800d-759589cec01a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:33.364259 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 995/1981] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:33 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:33.368566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab52626-88fe-491e-a183-6ea6a79b1ff3 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:33.393980 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab52626-88fe-491e-a183-6ea6a79b1ff3 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:33.457768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9fe85-176f-4ba9-800d-759589cec01a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:33.460544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9ef9fe85-176f-4ba9-800d-759589cec01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:33.580806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89f2d2-1baa-42da-9d03-cb12baa6bf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:33.581548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89f2d2-1baa-42da-9d03-cb12baa6bf07 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:33.581829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89f2d2-1baa-42da-9d03-cb12baa6bf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:33.582104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89f2d2-1baa-42da-9d03-cb12baa6bf07 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:33.587724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89f2d2-1baa-42da-9d03-cb12baa6bf07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:33.593067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89f2d2-1baa-42da-9d03-cb12baa6bf07 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:33.650271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 987/1982] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:33 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:33.653066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89f2d2-1baa-42da-9d03-cb12baa6bf07 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:33.678361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:33.683533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9fe85-176f-4ba9-800d-759589cec01a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:33.690846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89f2d2-1baa-42da-9d03-cb12baa6bf07 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:33.698273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:33.699183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:33.699478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:33.699762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] PATH_INFO: `/v3/projects/18e03663608447d0a0e4e9d4ec244b60` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:33.705775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18e03663608447d0a0e4e9d4ec244b60)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:33.710038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:33.713368 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778997b-acfd-4cd9-bcf1-0932113dc30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:33.714094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778997b-acfd-4cd9-bcf1-0932113dc30b admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:33.714571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778997b-acfd-4cd9-bcf1-0932113dc30b admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:33.714714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778997b-acfd-4cd9-bcf1-0932113dc30b admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:33.715467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778997b-acfd-4cd9-bcf1-0932113dc30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:33.719627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778997b-acfd-4cd9-bcf1-0932113dc30b admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:33.746424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 988/1983] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:33.751333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778997b-acfd-4cd9-bcf1-0932113dc30b admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:33.779419 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778997b-acfd-4cd9-bcf1-0932113dc30b admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:33.809421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bf52d-a4b9-499a-ac77-4467e6ecf25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:33.810922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bf52d-a4b9-499a-ac77-4467e6ecf25e admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:33.811402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bf52d-a4b9-499a-ac77-4467e6ecf25e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:33.811962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bf52d-a4b9-499a-ac77-4467e6ecf25e admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:33.812978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bf52d-a4b9-499a-ac77-4467e6ecf25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:33.817175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bf52d-a4b9-499a-ac77-4467e6ecf25e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:33.840178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18e03663608447d0a0e4e9d4ec244b60', 'request_id': 'req-15c2844c-35cb-419a-be82-9a45203aec50'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:33.840877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 989/1984] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:33.845915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:33.848341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bf52d-a4b9-499a-ac77-4467e6ecf25e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:33.868599 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bf52d-a4b9-499a-ac77-4467e6ecf25e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:33.873115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18e03663608447d0a0e4e9d4ec244b60', 'request_id': 'req-15c2844c-35cb-419a-be82-9a45203aec50'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:33.881800 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18e03663608447d0a0e4e9d4ec244b60', 'request_id': 'req-15c2844c-35cb-419a-be82-9a45203aec50'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:33.896706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] The token cache is being invalidate because project 18e03663608447d0a0e4e9d4ec244b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:33.896994 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:33.897977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-15c2844c-35cb-419a-be82-9a45203aec50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:33.900231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 996/1985] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:33 2022] DELETE /identity/v3/projects/18e03663608447d0a0e4e9d4ec244b60 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:33.904976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f482ce5-123b-4a84-a014-32a0095a0de1 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:33.906441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f482ce5-123b-4a84-a014-32a0095a0de1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:33.906950 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f482ce5-123b-4a84-a014-32a0095a0de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:33.933342 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-7f482ce5-123b-4a84-a014-32a0095a0de1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:33.934669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7f482ce5-123b-4a84-a014-32a0095a0de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:33.945791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab837875-66d1-4c33-93b0-a413fa346a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:33.946821 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab837875-66d1-4c33-93b0-a413fa346a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:33.947178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab837875-66d1-4c33-93b0-a413fa346a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:33.947557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab837875-66d1-4c33-93b0-a413fa346a43 admin admin] PATH_INFO: `/v3/projects/eff88c6329aa40c88e644f53197a9b46/users/5999914e5da0491aaae0c1c5a365c746/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:33.968606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab837875-66d1-4c33-93b0-a413fa346a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff88c6329aa40c88e644f53197a9b46, user_id=5999914e5da0491aaae0c1c5a365c746, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:33.971841 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab837875-66d1-4c33-93b0-a413fa346a43 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:33.990911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 990/1986] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:33 2022] PUT /identity/v3/projects/eff88c6329aa40c88e644f53197a9b46/users/5999914e5da0491aaae0c1c5a365c746/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:34.003580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f88c-a343-4994-959b-19be8366c6f5 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.003831 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f88c-a343-4994-959b-19be8366c6f5 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.004246 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f88c-a343-4994-959b-19be8366c6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.019548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 997/1987] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:34.029936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9c0c5-83cd-42e4-998c-d7645a81e6bc None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.030593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9c0c5-83cd-42e4-998c-d7645a81e6bc None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.031144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9c0c5-83cd-42e4-998c-d7645a81e6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.035528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-97e4f88c-a343-4994-959b-19be8366c6f5 None None] MFA Rules not processed for user `5999914e5da0491aaae0c1c5a365c746`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:34.037399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-97e4f88c-a343-4994-959b-19be8366c6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:34.053995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-8de9c0c5-83cd-42e4-998c-d7645a81e6bc None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:34.055325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8de9c0c5-83cd-42e4-998c-d7645a81e6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:34.114089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 991/1988] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:34.123268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 998/1989] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:34.128414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4f88c-a343-4994-959b-19be8366c6f5 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:34.128748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9c0c5-83cd-42e4-998c-d7645a81e6bc None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:34.129873 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-97e4f88c-a343-4994-959b-19be8366c6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:34.139051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9c0c5-83cd-42e4-998c-d7645a81e6bc None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:34.144715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e06d4-fcce-4da8-808f-d26ab0fbd197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:34.145418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e06d4-fcce-4da8-808f-d26ab0fbd197 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.145712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e06d4-fcce-4da8-808f-d26ab0fbd197 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.146384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e06d4-fcce-4da8-808f-d26ab0fbd197 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.147223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e06d4-fcce-4da8-808f-d26ab0fbd197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:34.147537 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e06d4-fcce-4da8-808f-d26ab0fbd197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:34.149243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e06d4-fcce-4da8-808f-d26ab0fbd197 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:34.162134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 999/1990] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:55:34.166507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e06d4-fcce-4da8-808f-d26ab0fbd197 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:34.180185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e06d4-fcce-4da8-808f-d26ab0fbd197 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:34.185306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710dccf-8c87-432e-a35b-4c5e0743252a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:34.185968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710dccf-8c87-432e-a35b-4c5e0743252a admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.186250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710dccf-8c87-432e-a35b-4c5e0743252a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.186527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710dccf-8c87-432e-a35b-4c5e0743252a admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.187218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710dccf-8c87-432e-a35b-4c5e0743252a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:34.189910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710dccf-8c87-432e-a35b-4c5e0743252a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:34.198428 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4f88c-a343-4994-959b-19be8366c6f5 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:34.202381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74840f-9918-48c8-ab9b-708897879a04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:34.203778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74840f-9918-48c8-ab9b-708897879a04 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.204133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74840f-9918-48c8-ab9b-708897879a04 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.204995 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74840f-9918-48c8-ab9b-708897879a04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.216728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74840f-9918-48c8-ab9b-708897879a04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:34.218822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74840f-9918-48c8-ab9b-708897879a04 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:34.223143 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:34.228973 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1000/1991] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:34 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:55:34.233010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710dccf-8c87-432e-a35b-4c5e0743252a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:34.233683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 992/1992] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:34.253473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74840f-9918-48c8-ab9b-708897879a04 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:34.254914 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3a74840f-9918-48c8-ab9b-708897879a04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:34.270402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74840f-9918-48c8-ab9b-708897879a04 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:34.291152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:34.318105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710dccf-8c87-432e-a35b-4c5e0743252a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:34.323414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111da118-a41f-4af9-a8f1-19927738177b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:34.324090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111da118-a41f-4af9-a8f1-19927738177b admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.324413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111da118-a41f-4af9-a8f1-19927738177b admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.324688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111da118-a41f-4af9-a8f1-19927738177b admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.325392 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111da118-a41f-4af9-a8f1-19927738177b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:34.327310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111da118-a41f-4af9-a8f1-19927738177b admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:34.340458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db479b1-e14a-4d30-a674-539ef30241e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:34.341470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db479b1-e14a-4d30-a674-539ef30241e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.341798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db479b1-e14a-4d30-a674-539ef30241e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.342107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db479b1-e14a-4d30-a674-539ef30241e7 admin admin] PATH_INFO: `/v3/users/5999914e5da0491aaae0c1c5a365c746` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.344799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db479b1-e14a-4d30-a674-539ef30241e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5999914e5da0491aaae0c1c5a365c746)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:34.346739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db479b1-e14a-4d30-a674-539ef30241e7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:34.353933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1001/1993] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:34 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:34.355134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111da118-a41f-4af9-a8f1-19927738177b admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:34.365471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111da118-a41f-4af9-a8f1-19927738177b admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:34.370939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59c2db-5338-482a-a2e9-a21b19960c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:34.371612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59c2db-5338-482a-a2e9-a21b19960c89 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.371942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59c2db-5338-482a-a2e9-a21b19960c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.372164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59c2db-5338-482a-a2e9-a21b19960c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.372939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59c2db-5338-482a-a2e9-a21b19960c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:34.374576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59c2db-5338-482a-a2e9-a21b19960c89 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:34.388839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1002/1994] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:55:34.389889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59c2db-5338-482a-a2e9-a21b19960c89 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:34.400764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59c2db-5338-482a-a2e9-a21b19960c89 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:34.400932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6db479b1-e14a-4d30-a674-539ef30241e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5999914e5da0491aaae0c1c5a365c746'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:34.405452 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f11342-3595-4d3e-ad6b-6f35c39ddf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:34.406167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f11342-3595-4d3e-ad6b-6f35c39ddf04 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.406455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f11342-3595-4d3e-ad6b-6f35c39ddf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.406729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f11342-3595-4d3e-ad6b-6f35c39ddf04 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.407476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f11342-3595-4d3e-ad6b-6f35c39ddf04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:34.408887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6db479b1-e14a-4d30-a674-539ef30241e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5999914e5da0491aaae0c1c5a365c746'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:34.409119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f11342-3595-4d3e-ad6b-6f35c39ddf04 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:34.416899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6db479b1-e14a-4d30-a674-539ef30241e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5999914e5da0491aaae0c1c5a365c746'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:34.417932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6db479b1-e14a-4d30-a674-539ef30241e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5999914e5da0491aaae0c1c5a365c746'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:34.419812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1003/1995] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:34.424066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f11342-3595-4d3e-ad6b-6f35c39ddf04 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:34.425887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-57f11342-3595-4d3e-ad6b-6f35c39ddf04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:34.430613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6db479b1-e14a-4d30-a674-539ef30241e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5999914e5da0491aaae0c1c5a365c746'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:34.436779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 993/1996] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:34 2022] DELETE /identity/v3/users/5999914e5da0491aaae0c1c5a365c746 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:34.439387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f11342-3595-4d3e-ad6b-6f35c39ddf04 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:34.442016 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333337c-7594-492e-a477-b98ec74731c5 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.442303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333337c-7594-492e-a477-b98ec74731c5 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.442580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333337c-7594-492e-a477-b98ec74731c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.462578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-6333337c-7594-492e-a477-b98ec74731c5 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:34.463791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6333337c-7594-492e-a477-b98ec74731c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:34.500556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c390a5c-735b-4ff1-a8ff-73e3d89ab24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:34.501470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390a5c-735b-4ff1-a8ff-73e3d89ab24e admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.501779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390a5c-735b-4ff1-a8ff-73e3d89ab24e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.502056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390a5c-735b-4ff1-a8ff-73e3d89ab24e admin admin] PATH_INFO: `/v3/projects/0402894d9a08472aae07d5c07ff0c966/users/7242a599a607400eae2dd49c7cbf07e4/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.518970 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c390a5c-735b-4ff1-a8ff-73e3d89ab24e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0402894d9a08472aae07d5c07ff0c966, user_id=7242a599a607400eae2dd49c7cbf07e4, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:34.521935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c390a5c-735b-4ff1-a8ff-73e3d89ab24e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:34.524598 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 994/1997] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:34.534586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333337c-7594-492e-a477-b98ec74731c5 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:34.535985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6333337c-7594-492e-a477-b98ec74731c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:34.536715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1004/1998] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:34 2022] PUT /identity/v3/projects/0402894d9a08472aae07d5c07ff0c966/users/7242a599a607400eae2dd49c7cbf07e4/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:34.549054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ab4c7-7181-4833-b7f0-2553c3e6cf37 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.549353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ab4c7-7181-4833-b7f0-2553c3e6cf37 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.549717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ab4c7-7181-4833-b7f0-2553c3e6cf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.561912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:34.582244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-707ab4c7-7181-4833-b7f0-2553c3e6cf37 None None] MFA Rules not processed for user `7242a599a607400eae2dd49c7cbf07e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:34.583551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-707ab4c7-7181-4833-b7f0-2553c3e6cf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:34.606196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333337c-7594-492e-a477-b98ec74731c5 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:34.670712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1005/1999] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:34.682241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ab4c7-7181-4833-b7f0-2553c3e6cf37 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:34.690171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580906f-b36f-4517-8dce-c1b380739cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:34.691394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580906f-b36f-4517-8dce-c1b380739cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.692018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580906f-b36f-4517-8dce-c1b380739cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.692504 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580906f-b36f-4517-8dce-c1b380739cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.727200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:34.794709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ab4c7-7181-4833-b7f0-2553c3e6cf37 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:34.796040 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580906f-b36f-4517-8dce-c1b380739cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:34.799548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580906f-b36f-4517-8dce-c1b380739cb4 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:34.802528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9606d40-659b-439e-ad9c-940f8752577b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:34.803518 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9606d40-659b-439e-ad9c-940f8752577b admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:34.803907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9606d40-659b-439e-ad9c-940f8752577b admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:34.804252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9606d40-659b-439e-ad9c-940f8752577b admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:34.805240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9606d40-659b-439e-ad9c-940f8752577b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:34.808583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9606d40-659b-439e-ad9c-940f8752577b admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:34.820217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 995/2000] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:34.860676 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1006/2001] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:34 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:34.865607 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580906f-b36f-4517-8dce-c1b380739cb4 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:34.879002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580906f-b36f-4517-8dce-c1b380739cb4 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:35.033173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd71ff-8c87-4aa6-8915-be17749ef24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:35.034506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd71ff-8c87-4aa6-8915-be17749ef24d admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:35.035163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd71ff-8c87-4aa6-8915-be17749ef24d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:35.035752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd71ff-8c87-4aa6-8915-be17749ef24d admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:35.037514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd71ff-8c87-4aa6-8915-be17749ef24d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:35.040222 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd71ff-8c87-4aa6-8915-be17749ef24d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:35.067948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 996/2002] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:34 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:35.069201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9606d40-659b-439e-ad9c-940f8752577b admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:35.083087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9606d40-659b-439e-ad9c-940f8752577b admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:35.090343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d181c-4c52-42b8-8bbd-d825839d6100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:35.091338 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d181c-4c52-42b8-8bbd-d825839d6100 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:35.092167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d181c-4c52-42b8-8bbd-d825839d6100 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:35.092772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d181c-4c52-42b8-8bbd-d825839d6100 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:35.094378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d181c-4c52-42b8-8bbd-d825839d6100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:35.097333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d181c-4c52-42b8-8bbd-d825839d6100 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:35.114065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1007/2003] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:35.120792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd71ff-8c87-4aa6-8915-be17749ef24d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:35.134233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd71ff-8c87-4aa6-8915-be17749ef24d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:35.144022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab035672-8585-44fe-908c-54223c64ff44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:35.145396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab035672-8585-44fe-908c-54223c64ff44 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:35.146649 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab035672-8585-44fe-908c-54223c64ff44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:35.146871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab035672-8585-44fe-908c-54223c64ff44 admin admin] PATH_INFO: `/v3/projects/d8abe092b88a4ca8adb8d7ab84bf6468/users/36eb245700294f9889926d68adfa483e/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:35.164062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:35.169470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab035672-8585-44fe-908c-54223c64ff44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8abe092b88a4ca8adb8d7ab84bf6468, user_id=36eb245700294f9889926d68adfa483e, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:35.173469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab035672-8585-44fe-908c-54223c64ff44 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:35.229384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 997/2004] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:35 2022] PUT /identity/v3/projects/d8abe092b88a4ca8adb8d7ab84bf6468/users/36eb245700294f9889926d68adfa483e/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:55:35.234794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d181c-4c52-42b8-8bbd-d825839d6100 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:35.273907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d181c-4c52-42b8-8bbd-d825839d6100 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:35.416142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f8a47-aeb0-4ac9-beec-c4a8c6f3ab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:35.416607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f8a47-aeb0-4ac9-beec-c4a8c6f3ab4e admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:35.416901 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f8a47-aeb0-4ac9-beec-c4a8c6f3ab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:35.417185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f8a47-aeb0-4ac9-beec-c4a8c6f3ab4e admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:35.418393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f8a47-aeb0-4ac9-beec-c4a8c6f3ab4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:35.420455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f8a47-aeb0-4ac9-beec-c4a8c6f3ab4e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:35.437275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1008/2005] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:35.443927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab035672-8585-44fe-908c-54223c64ff44 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:35.457147 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab035672-8585-44fe-908c-54223c64ff44 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:35.472714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad524af-ad1d-4812-9897-a151d4af7742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:35.473806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad524af-ad1d-4812-9897-a151d4af7742 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:35.474098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad524af-ad1d-4812-9897-a151d4af7742 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:35.474381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad524af-ad1d-4812-9897-a151d4af7742 admin admin] PATH_INFO: `/v3/projects/d8abe092b88a4ca8adb8d7ab84bf6468/users/36eb245700294f9889926d68adfa483e/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:35.476491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad524af-ad1d-4812-9897-a151d4af7742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8abe092b88a4ca8adb8d7ab84bf6468, user_id=36eb245700294f9889926d68adfa483e, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:35.479144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad524af-ad1d-4812-9897-a151d4af7742 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:35.506940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 998/2006] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:35 2022] PUT /identity/v3/projects/d8abe092b88a4ca8adb8d7ab84bf6468/users/36eb245700294f9889926d68adfa483e/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:35.518600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203d66e-9647-4f75-ab2a-29c82f214828 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:35.518894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203d66e-9647-4f75-ab2a-29c82f214828 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:35.519601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203d66e-9647-4f75-ab2a-29c82f214828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:35.558613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-6203d66e-9647-4f75-ab2a-29c82f214828 None None] MFA Rules not processed for user `36eb245700294f9889926d68adfa483e`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:35.559934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6203d66e-9647-4f75-ab2a-29c82f214828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:35.589429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad524af-ad1d-4812-9897-a151d4af7742 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:35.592043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8ad524af-ad1d-4812-9897-a151d4af7742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:35.660031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1009/2007] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:35.672796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203d66e-9647-4f75-ab2a-29c82f214828 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:35.674330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6203d66e-9647-4f75-ab2a-29c82f214828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:35.680356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad524af-ad1d-4812-9897-a151d4af7742 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:35.685669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:35.686543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:35.686826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:35.687169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] PATH_INFO: `/v3/projects/eff88c6329aa40c88e644f53197a9b46` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:35.688513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff88c6329aa40c88e644f53197a9b46)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:35.690654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:35.704350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:35.754749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff88c6329aa40c88e644f53197a9b46', 'request_id': 'req-271108e3-6517-45b0-b6db-93673c1a6178'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:35.759690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203d66e-9647-4f75-ab2a-29c82f214828 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:35.765196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff88c6329aa40c88e644f53197a9b46', 'request_id': 'req-271108e3-6517-45b0-b6db-93673c1a6178'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:35.772830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff88c6329aa40c88e644f53197a9b46', 'request_id': 'req-271108e3-6517-45b0-b6db-93673c1a6178'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:35.784974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] The token cache is being invalidate because project eff88c6329aa40c88e644f53197a9b46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:35.785293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:35.786428 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-271108e3-6517-45b0-b6db-93673c1a6178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:35.791500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 999/2008] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:35 2022] DELETE /identity/v3/projects/eff88c6329aa40c88e644f53197a9b46 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:35.793020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e91c4-e093-402c-acba-9fcd72a18353 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:35.793312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e91c4-e093-402c-acba-9fcd72a18353 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:35.793968 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e91c4-e093-402c-acba-9fcd72a18353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:35.816310 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-713e91c4-e093-402c-acba-9fcd72a18353 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:35.817640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-713e91c4-e093-402c-acba-9fcd72a18353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:35.827767 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb631a-0e00-4c15-b9d0-2a4b50bb09d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:35.828581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb631a-0e00-4c15-b9d0-2a4b50bb09d8 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:35.828867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb631a-0e00-4c15-b9d0-2a4b50bb09d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:35.829201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb631a-0e00-4c15-b9d0-2a4b50bb09d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:35.831992 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7adb631a-0e00-4c15-b9d0-2a4b50bb09d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:35.892654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1000/2009] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:35.901474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b3e6-f319-41e8-b1c4-00fa919ab6c7 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:35.901777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b3e6-f319-41e8-b1c4-00fa919ab6c7 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:35.902038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b3e6-f319-41e8-b1c4-00fa919ab6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:35.910177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb631a-0e00-4c15-b9d0-2a4b50bb09d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:35.911524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb631a-0e00-4c15-b9d0-2a4b50bb09d8 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:35.916186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:35.923544 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-9949b3e6-f319-41e8-b1c4-00fa919ab6c7 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:35.925910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-9949b3e6-f319-41e8-b1c4-00fa919ab6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:35.927648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1010/2010] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:55:35.940100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1001/2011] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:35.940982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb631a-0e00-4c15-b9d0-2a4b50bb09d8 service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:35.960810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb631a-0e00-4c15-b9d0-2a4b50bb09d8 service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:35.966743 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b1795-8049-44d2-9724-6bbe924d7610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:35.967427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b1795-8049-44d2-9724-6bbe924d7610 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:35.967717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b1795-8049-44d2-9724-6bbe924d7610 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:35.967998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b1795-8049-44d2-9724-6bbe924d7610 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:35.969302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b1795-8049-44d2-9724-6bbe924d7610 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:35.969721 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b1795-8049-44d2-9724-6bbe924d7610 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:35.972598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b1795-8049-44d2-9724-6bbe924d7610 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:35.989791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1011/2012] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:55:35.994048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949b3e6-f319-41e8-b1c4-00fa919ab6c7 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:36.004952 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949b3e6-f319-41e8-b1c4-00fa919ab6c7 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:36.040988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a85003-b074-4d44-9344-251420effd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:36.042022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a85003-b074-4d44-9344-251420effd07 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:36.043117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a85003-b074-4d44-9344-251420effd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:36.043851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a85003-b074-4d44-9344-251420effd07 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:36.045651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a85003-b074-4d44-9344-251420effd07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:36.051562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a85003-b074-4d44-9344-251420effd07 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:36.103070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1002/2013] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:35 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:36.106876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b1795-8049-44d2-9724-6bbe924d7610 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:36.118657 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b1795-8049-44d2-9724-6bbe924d7610 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:36.258536 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3ba8d-4cac-46a0-87c1-b05f41bcd3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:36.259582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3ba8d-4cac-46a0-87c1-b05f41bcd3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:36.259582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3ba8d-4cac-46a0-87c1-b05f41bcd3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:36.259933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3ba8d-4cac-46a0-87c1-b05f41bcd3a4 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:36.261163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3ba8d-4cac-46a0-87c1-b05f41bcd3a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:36.262852 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3ba8d-4cac-46a0-87c1-b05f41bcd3a4 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:36.303867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1012/2014] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:36 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:36.305508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a85003-b074-4d44-9344-251420effd07 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:36.319654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a85003-b074-4d44-9344-251420effd07 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:36.333279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193d1d6-b4c4-41e2-80bb-1bcc14e16a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:36.334312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193d1d6-b4c4-41e2-80bb-1bcc14e16a7f admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:36.334686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193d1d6-b4c4-41e2-80bb-1bcc14e16a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:36.334973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193d1d6-b4c4-41e2-80bb-1bcc14e16a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:36.335730 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193d1d6-b4c4-41e2-80bb-1bcc14e16a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:36.341185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193d1d6-b4c4-41e2-80bb-1bcc14e16a7f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:36.374802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1003/2015] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:36.377122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3ba8d-4cac-46a0-87c1-b05f41bcd3a4 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:36.402840 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3ba8d-4cac-46a0-87c1-b05f41bcd3a4 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:36.412765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64186f4d-af96-4227-9840-4b9f6be24d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:36.414207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64186f4d-af96-4227-9840-4b9f6be24d8b admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:36.415913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64186f4d-af96-4227-9840-4b9f6be24d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:36.417338 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64186f4d-af96-4227-9840-4b9f6be24d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:36.418918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64186f4d-af96-4227-9840-4b9f6be24d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:36.420564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64186f4d-af96-4227-9840-4b9f6be24d8b admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:36.422801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:36.466233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1013/2016] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:55:36.469941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193d1d6-b4c4-41e2-80bb-1bcc14e16a7f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:36.488178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193d1d6-b4c4-41e2-80bb-1bcc14e16a7f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:36.504546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8921df-5e34-47c0-8b95-8fe4e0e25c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:36.508360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8921df-5e34-47c0-8b95-8fe4e0e25c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:36.510226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8921df-5e34-47c0-8b95-8fe4e0e25c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:36.510571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8921df-5e34-47c0-8b95-8fe4e0e25c41 admin admin] PATH_INFO: `/v3/projects/a05f2ab99a1b4a6b9190ac7b513f7896/users/6688fdba27694ea0bc3e208e30cd2a26/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:36.557139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8921df-5e34-47c0-8b95-8fe4e0e25c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05f2ab99a1b4a6b9190ac7b513f7896, user_id=6688fdba27694ea0bc3e208e30cd2a26, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:36.564708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8921df-5e34-47c0-8b95-8fe4e0e25c41 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:36.604549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1004/2017] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:55:36 2022] PUT /identity/v3/projects/a05f2ab99a1b4a6b9190ac7b513f7896/users/6688fdba27694ea0bc3e208e30cd2a26/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:36.612906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cfb66-3e88-498f-9e07-8af6cfd7f591 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:36.613210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cfb66-3e88-498f-9e07-8af6cfd7f591 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:36.613545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cfb66-3e88-498f-9e07-8af6cfd7f591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:36.666339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-242cfb66-3e88-498f-9e07-8af6cfd7f591 None None] MFA Rules not processed for user `6688fdba27694ea0bc3e208e30cd2a26`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:36.667592 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-242cfb66-3e88-498f-9e07-8af6cfd7f591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:36.845811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1014/2018] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:55:36.860087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8921df-5e34-47c0-8b95-8fe4e0e25c41 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:36.863770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2b8921df-5e34-47c0-8b95-8fe4e0e25c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:37.039570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8921df-5e34-47c0-8b95-8fe4e0e25c41 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:37.045348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95179c6d-9b9b-4619-8f9d-74d9a695dd30 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35e7720e3bf4c9e80bdefa6a1cb8296', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:37.046036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95179c6d-9b9b-4619-8f9d-74d9a695dd30 service glance] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:37.046324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95179c6d-9b9b-4619-8f9d-74d9a695dd30 service glance] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:37.047350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95179c6d-9b9b-4619-8f9d-74d9a695dd30 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:37.067868 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95179c6d-9b9b-4619-8f9d-74d9a695dd30 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:37.070038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95179c6d-9b9b-4619-8f9d-74d9a695dd30 service glance] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:37.092211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1005/2019] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:44.263802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cfb66-3e88-498f-9e07-8af6cfd7f591 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:44.265712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-242cfb66-3e88-498f-9e07-8af6cfd7f591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:44.333782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cfb66-3e88-498f-9e07-8af6cfd7f591 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:44.337744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a6e47-04f3-4cf6-a516-883a96975bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:44.338396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a6e47-04f3-4cf6-a516-883a96975bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:44.338818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a6e47-04f3-4cf6-a516-883a96975bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:44.339098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a6e47-04f3-4cf6-a516-883a96975bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:44.340951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-de5a6e47-04f3-4cf6-a516-883a96975bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:44.412565 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a6e47-04f3-4cf6-a516-883a96975bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:44.414002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a6e47-04f3-4cf6-a516-883a96975bb1 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:44.425474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1015/2020] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:47.082041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=59,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:54.420397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=84 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:57.971861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95179c6d-9b9b-4619-8f9d-74d9a695dd30 service glance] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:57.973439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-95179c6d-9b9b-4619-8f9d-74d9a695dd30 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:58.046724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95179c6d-9b9b-4619-8f9d-74d9a695dd30 service glance] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.051361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.052237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.052550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.052827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] PATH_INFO: `/v3/users/e31ac10534c74888ae6a9f71b840af1c` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.054058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e31ac10534c74888ae6a9f71b840af1c)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.055979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:58.109268 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e31ac10534c74888ae6a9f71b840af1c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.118451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e31ac10534c74888ae6a9f71b840af1c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.128269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e31ac10534c74888ae6a9f71b840af1c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.129192 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e31ac10534c74888ae6a9f71b840af1c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.141829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e31ac10534c74888ae6a9f71b840af1c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.147851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1006/2021] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:57 2022] DELETE /identity/v3/users/e31ac10534c74888ae6a9f71b840af1c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:58.152411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d118b-488e-4514-b1db-5c1069b6aec9 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.153245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d118b-488e-4514-b1db-5c1069b6aec9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.153525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d118b-488e-4514-b1db-5c1069b6aec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.173530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-7c1d118b-488e-4514-b1db-5c1069b6aec9 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:58.174769 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7c1d118b-488e-4514-b1db-5c1069b6aec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:58.249571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1016/2022] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:58.259486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:58.261074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:58.323939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c1159-5c61-427e-9979-01b42bc8e00e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.327822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea70cf-4de2-430a-9ff9-fc0d7dec3a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.328519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea70cf-4de2-430a-9ff9-fc0d7dec3a3c service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.328805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea70cf-4de2-430a-9ff9-fc0d7dec3a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.329083 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea70cf-4de2-430a-9ff9-fc0d7dec3a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.338797 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea70cf-4de2-430a-9ff9-fc0d7dec3a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.340226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea70cf-4de2-430a-9ff9-fc0d7dec3a3c service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:58.351068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1007/2023] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:55:58.648701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d118b-488e-4514-b1db-5c1069b6aec9 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:58.651609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7c1d118b-488e-4514-b1db-5c1069b6aec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:58.699047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d118b-488e-4514-b1db-5c1069b6aec9 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.705456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.706269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.706556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.706841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] PATH_INFO: `/v3/projects/78fe5b511fce440799085a216415371a` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.708089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78fe5b511fce440799085a216415371a)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.710222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:58.778518 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78fe5b511fce440799085a216415371a', 'request_id': 'req-a0075112-59d6-4ad4-800c-262db65d3401'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.788276 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78fe5b511fce440799085a216415371a', 'request_id': 'req-a0075112-59d6-4ad4-800c-262db65d3401'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.795820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78fe5b511fce440799085a216415371a', 'request_id': 'req-a0075112-59d6-4ad4-800c-262db65d3401'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.808997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] The token cache is being invalidate because project 78fe5b511fce440799085a216415371a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:58.809253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.809952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:58.811604 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1017/2024] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:58 2022] DELETE /identity/v3/projects/78fe5b511fce440799085a216415371a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:58.816451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea70cf-4de2-430a-9ff9-fc0d7dec3a3c service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:58.829442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea70cf-4de2-430a-9ff9-fc0d7dec3a3c service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:58.854993 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:58.909240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71dad5-43bf-4273-8762-3da04ecf6b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:58.910623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71dad5-43bf-4273-8762-3da04ecf6b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:58.911052 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71dad5-43bf-4273-8762-3da04ecf6b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:58.911584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71dad5-43bf-4273-8762-3da04ecf6b7d admin admin] PATH_INFO: `/v3/users/ad3bc9cfff0e4797906a0c0a496ea484` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:58.915692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71dad5-43bf-4273-8762-3da04ecf6b7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad3bc9cfff0e4797906a0c0a496ea484)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:58.919396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71dad5-43bf-4273-8762-3da04ecf6b7d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.005126 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-db71dad5-43bf-4273-8762-3da04ecf6b7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad3bc9cfff0e4797906a0c0a496ea484'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:59.017941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-db71dad5-43bf-4273-8762-3da04ecf6b7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad3bc9cfff0e4797906a0c0a496ea484'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:59.027800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-db71dad5-43bf-4273-8762-3da04ecf6b7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad3bc9cfff0e4797906a0c0a496ea484'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:59.028592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-db71dad5-43bf-4273-8762-3da04ecf6b7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad3bc9cfff0e4797906a0c0a496ea484'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:59.042365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-db71dad5-43bf-4273-8762-3da04ecf6b7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad3bc9cfff0e4797906a0c0a496ea484'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:59.051047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1008/2025] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:55:58 2022] DELETE /identity/v3/users/ad3bc9cfff0e4797906a0c0a496ea484 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:59.420197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:59.422329 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:59.436492 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0075112-59d6-4ad4-800c-262db65d3401 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:59.509983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:59.510858 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.511141 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.511414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] PATH_INFO: `/v3/projects/4a853ed2954f407abe423880763da34a` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.512677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a853ed2954f407abe423880763da34a)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:59.514583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.557637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:59.585116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a853ed2954f407abe423880763da34a', 'request_id': 'req-9f933541-ea23-4550-b98d-9b8e51e8fad6'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:59.594020 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a853ed2954f407abe423880763da34a', 'request_id': 'req-9f933541-ea23-4550-b98d-9b8e51e8fad6'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:59.601551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a853ed2954f407abe423880763da34a', 'request_id': 'req-9f933541-ea23-4550-b98d-9b8e51e8fad6'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:59.613891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] The token cache is being invalidate because project 4a853ed2954f407abe423880763da34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:55:59.614235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:59.614905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9f933541-ea23-4550-b98d-9b8e51e8fad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:55:59.616815 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1018/2026] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:55:59 2022] DELETE /identity/v3/projects/4a853ed2954f407abe423880763da34a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:55:59.622792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc994f9c-a737-4c00-be42-78ced8bc18f2 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.623184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc994f9c-a737-4c00-be42-78ced8bc18f2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.623526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc994f9c-a737-4c00-be42-78ced8bc18f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.646799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-fc994f9c-a737-4c00-be42-78ced8bc18f2 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:59.648154 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fc994f9c-a737-4c00-be42-78ced8bc18f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:59.714633 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1009/2027] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:59.724155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d642bf-e7df-4147-89eb-9ab3337ef995 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.724504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d642bf-e7df-4147-89eb-9ab3337ef995 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.725052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d642bf-e7df-4147-89eb-9ab3337ef995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.745732 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-21d642bf-e7df-4147-89eb-9ab3337ef995 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:59.747042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-21d642bf-e7df-4147-89eb-9ab3337ef995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:59.757822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1019/2028] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:55:59.762298 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc994f9c-a737-4c00-be42-78ced8bc18f2 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:59.772302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc994f9c-a737-4c00-be42-78ced8bc18f2 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:59.777091 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf17f46-d64d-4e96-a964-7cf4311564d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:59.777857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf17f46-d64d-4e96-a964-7cf4311564d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.778135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf17f46-d64d-4e96-a964-7cf4311564d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.778417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf17f46-d64d-4e96-a964-7cf4311564d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.779227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf17f46-d64d-4e96-a964-7cf4311564d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:55:59.779537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf17f46-d64d-4e96-a964-7cf4311564d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:59.781717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf17f46-d64d-4e96-a964-7cf4311564d6 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.799484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1010/2029] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:55:59.807710 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d642bf-e7df-4147-89eb-9ab3337ef995 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:59.817855 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d642bf-e7df-4147-89eb-9ab3337ef995 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:59.823487 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd27ff1-230a-40a0-931c-61218ba11afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:59.824210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd27ff1-230a-40a0-931c-61218ba11afd admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.824570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd27ff1-230a-40a0-931c-61218ba11afd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.824882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd27ff1-230a-40a0-931c-61218ba11afd admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.825649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd27ff1-230a-40a0-931c-61218ba11afd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:59.827727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd27ff1-230a-40a0-931c-61218ba11afd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.862567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1020/2030] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:55:59 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:59.867027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf17f46-d64d-4e96-a964-7cf4311564d6 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:55:59.878398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf17f46-d64d-4e96-a964-7cf4311564d6 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:55:59.910920 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:55:59.920253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b367f38-e096-40dd-8864-55e3b15d66c9 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.920586 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b367f38-e096-40dd-8864-55e3b15d66c9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.920867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b367f38-e096-40dd-8864-55e3b15d66c9 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.922857 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1021/2031] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 20:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 20:55:59.940816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb839f-1ae8-4dde-9add-158653b3e7ba None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.941134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb839f-1ae8-4dde-9add-158653b3e7ba None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.941403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb839f-1ae8-4dde-9add-158653b3e7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.951928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52655714-c107-4085-8fbe-627e64429c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:55:59.954328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52655714-c107-4085-8fbe-627e64429c1e admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:55:59.954750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52655714-c107-4085-8fbe-627e64429c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:55:59.955261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52655714-c107-4085-8fbe-627e64429c1e admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:55:59.956111 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52655714-c107-4085-8fbe-627e64429c1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:55:59.958350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52655714-c107-4085-8fbe-627e64429c1e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:55:59.961375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-bceb839f-1ae8-4dde-9add-158653b3e7ba None None] MFA Rules not processed for user `fffebfc51dba47f68263c0bcd6b18d22`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:55:59.963315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-bceb839f-1ae8-4dde-9add-158653b3e7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:55:59.991874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1011/2032] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:55:59 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:55:59.993325 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52655714-c107-4085-8fbe-627e64429c1e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.007905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52655714-c107-4085-8fbe-627e64429c1e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.013265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aae1d6-f12b-4bc5-b456-74c89bef4346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.014049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aae1d6-f12b-4bc5-b456-74c89bef4346 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.015734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aae1d6-f12b-4bc5-b456-74c89bef4346 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.015734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aae1d6-f12b-4bc5-b456-74c89bef4346 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.016086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aae1d6-f12b-4bc5-b456-74c89bef4346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.018768 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aae1d6-f12b-4bc5-b456-74c89bef4346 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.034460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1012/2033] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:00.038255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aae1d6-f12b-4bc5-b456-74c89bef4346 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.052895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aae1d6-f12b-4bc5-b456-74c89bef4346 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.059000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef1080-1f3d-4270-8ea9-9d3f0ddb1777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.059243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1022/2034] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 20:55:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:00.059752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef1080-1f3d-4270-8ea9-9d3f0ddb1777 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.060129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef1080-1f3d-4270-8ea9-9d3f0ddb1777 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.060592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef1080-1f3d-4270-8ea9-9d3f0ddb1777 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.060840 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb839f-1ae8-4dde-9add-158653b3e7ba None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.061809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef1080-1f3d-4270-8ea9-9d3f0ddb1777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.062242 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-bceb839f-1ae8-4dde-9add-158653b3e7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.063710 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef1080-1f3d-4270-8ea9-9d3f0ddb1777 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.080791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1013/2035] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:00.081890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef1080-1f3d-4270-8ea9-9d3f0ddb1777 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.083717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-5aef1080-1f3d-4270-8ea9-9d3f0ddb1777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.092612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb839f-1ae8-4dde-9add-158653b3e7ba None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.098439 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.099331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.099808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.099977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] PATH_INFO: `/v3/users/b621ebc1564a4719a0b8aa147af2194f` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.100084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef1080-1f3d-4270-8ea9-9d3f0ddb1777 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.101605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b621ebc1564a4719a0b8aa147af2194f)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.103403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.105548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299511a-9c29-42df-a30b-a73c160f8301 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.106278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299511a-9c29-42df-a30b-a73c160f8301 service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.106737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299511a-9c29-42df-a30b-a73c160f8301 service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.107051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299511a-9c29-42df-a30b-a73c160f8301 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.118843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299511a-9c29-42df-a30b-a73c160f8301 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.121274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299511a-9c29-42df-a30b-a73c160f8301 service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.137585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1014/2036] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:00.138559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299511a-9c29-42df-a30b-a73c160f8301 service placement] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.153022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299511a-9c29-42df-a30b-a73c160f8301 service placement] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.159754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c02941-0bef-4503-92ea-3fd31fbc2e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.165741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c02941-0bef-4503-92ea-3fd31fbc2e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.166142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c02941-0bef-4503-92ea-3fd31fbc2e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.166524 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c02941-0bef-4503-92ea-3fd31fbc2e0a admin admin] PATH_INFO: `/v3/projects/63c8f7374da74cd8ae2d66e751a75484/users/1b9330b410f04d759124380d3c4f04bc/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.180725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b621ebc1564a4719a0b8aa147af2194f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.189339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c02941-0bef-4503-92ea-3fd31fbc2e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c8f7374da74cd8ae2d66e751a75484, user_id=1b9330b410f04d759124380d3c4f04bc, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.190486 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b621ebc1564a4719a0b8aa147af2194f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.191214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b621ebc1564a4719a0b8aa147af2194f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.192252 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c02941-0bef-4503-92ea-3fd31fbc2e0a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.211374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b621ebc1564a4719a0b8aa147af2194f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.211574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1015/2037] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:00 2022] PUT /identity/v3/projects/63c8f7374da74cd8ae2d66e751a75484/users/1b9330b410f04d759124380d3c4f04bc/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:00.215766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:00.223161 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d7c9b-9a64-474e-abec-cd61751277d7 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.223482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d7c9b-9a64-474e-abec-cd61751277d7 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.223735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d7c9b-9a64-474e-abec-cd61751277d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.226081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b621ebc1564a4719a0b8aa147af2194f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.233371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1023/2038] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:00 2022] DELETE /identity/v3/users/b621ebc1564a4719a0b8aa147af2194f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:00.254189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-b71d7c9b-9a64-474e-abec-cd61751277d7 None None] MFA Rules not processed for user `1b9330b410f04d759124380d3c4f04bc`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:00.255469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b71d7c9b-9a64-474e-abec-cd61751277d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.334265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1016/2039] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:00.339102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.340875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.418759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e9589-9cd1-45d5-894f-beb1997c5bd2 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.425509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d7c9b-9a64-474e-abec-cd61751277d7 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.427273 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b71d7c9b-9a64-474e-abec-cd61751277d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.427824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effcde5-b376-4f23-8732-cc1f1d166151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.428600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effcde5-b376-4f23-8732-cc1f1d166151 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.428986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effcde5-b376-4f23-8732-cc1f1d166151 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.429539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effcde5-b376-4f23-8732-cc1f1d166151 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.429890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effcde5-b376-4f23-8732-cc1f1d166151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.431859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effcde5-b376-4f23-8732-cc1f1d166151 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.444141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d7c9b-9a64-474e-abec-cd61751277d7 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.451034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.452039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.452530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.452925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] PATH_INFO: `/v3/projects/69986b31fd064e4f824856578e732a39` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.454468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69986b31fd064e4f824856578e732a39)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.461012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.478736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1024/2040] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:00 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:00.483368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effcde5-b376-4f23-8732-cc1f1d166151 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.516990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:00.525447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69986b31fd064e4f824856578e732a39', 'request_id': 'req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.535318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69986b31fd064e4f824856578e732a39', 'request_id': 'req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.542166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69986b31fd064e4f824856578e732a39', 'request_id': 'req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.553101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] The token cache is being invalidate because project 69986b31fd064e4f824856578e732a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:00.553391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.554050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.555017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effcde5-b376-4f23-8732-cc1f1d166151 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.555832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1017/2041] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:00 2022] DELETE /identity/v3/projects/69986b31fd064e4f824856578e732a39 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:00.559714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.562432 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.618252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff429c45-9684-412f-a7bb-8dada1d27271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.618934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff429c45-9684-412f-a7bb-8dada1d27271 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.619244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff429c45-9684-412f-a7bb-8dada1d27271 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.619540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff429c45-9684-412f-a7bb-8dada1d27271 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.620244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff429c45-9684-412f-a7bb-8dada1d27271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.622199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff429c45-9684-412f-a7bb-8dada1d27271 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.631488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c06f6c-3a95-4e98-89f6-fba3ed25bdc5 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.636834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.637820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.638102 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.638579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] PATH_INFO: `/v3/users/370c788de1cc481591137176bb4ed486` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.640697 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=370c788de1cc481591137176bb4ed486)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.643058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.646651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1025/2042] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:00 2022] POST /identity/v3/users => generated 300 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:56:00.650992 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff429c45-9684-412f-a7bb-8dada1d27271 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.653410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ff429c45-9684-412f-a7bb-8dada1d27271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.666330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff429c45-9684-412f-a7bb-8dada1d27271 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.671377 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17903-a2ce-422d-ae81-ff0bfc8e1a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.672057 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17903-a2ce-422d-ae81-ff0bfc8e1a52 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.672427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17903-a2ce-422d-ae81-ff0bfc8e1a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.672765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17903-a2ce-422d-ae81-ff0bfc8e1a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.673621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf17903-a2ce-422d-ae81-ff0bfc8e1a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.675442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf17903-a2ce-422d-ae81-ff0bfc8e1a52 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.688807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1026/2043] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:00.693649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17903-a2ce-422d-ae81-ff0bfc8e1a52 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.699986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:00.703794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370c788de1cc481591137176bb4ed486'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.705919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17903-a2ce-422d-ae81-ff0bfc8e1a52 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.713833 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffcaff-7650-435f-af66-6599e4258312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.714071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370c788de1cc481591137176bb4ed486'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.714927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffcaff-7650-435f-af66-6599e4258312 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.715291 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffcaff-7650-435f-af66-6599e4258312 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.715574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffcaff-7650-435f-af66-6599e4258312 admin admin] PATH_INFO: `/v3/projects/ffec7ef694654e0aab3c3fd0fa3bedf1/users/96ac1ad045b4494980a2bfc74f1aa27f/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.722124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370c788de1cc481591137176bb4ed486'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.722816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370c788de1cc481591137176bb4ed486'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.732427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffcaff-7650-435f-af66-6599e4258312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffec7ef694654e0aab3c3fd0fa3bedf1, user_id=96ac1ad045b4494980a2bfc74f1aa27f, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.736089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffcaff-7650-435f-af66-6599e4258312 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.737389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370c788de1cc481591137176bb4ed486'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:00.745144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1018/2044] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:00 2022] DELETE /identity/v3/users/370c788de1cc481591137176bb4ed486 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:00.756982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1027/2045] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:00 2022] PUT /identity/v3/projects/ffec7ef694654e0aab3c3fd0fa3bedf1/users/96ac1ad045b4494980a2bfc74f1aa27f/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:00.758820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.760289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.816812 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:00.836748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c29bca-9e19-4950-85f5-a067562ebac8 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.841681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbc822-a7d7-47be-833a-ab337e89c880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.842385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbc822-a7d7-47be-833a-ab337e89c880 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.842672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbc822-a7d7-47be-833a-ab337e89c880 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.842984 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbc822-a7d7-47be-833a-ab337e89c880 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.843739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbc822-a7d7-47be-833a-ab337e89c880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.845395 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbc822-a7d7-47be-833a-ab337e89c880 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.856737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1019/2046] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:00.861908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffcaff-7650-435f-af66-6599e4258312 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.937922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffcaff-7650-435f-af66-6599e4258312 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:00.944613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc4f15-4629-49eb-bb70-3530518b1483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:00.945547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc4f15-4629-49eb-bb70-3530518b1483 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.945832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc4f15-4629-49eb-bb70-3530518b1483 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.946110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc4f15-4629-49eb-bb70-3530518b1483 admin admin] PATH_INFO: `/v3/projects/ffec7ef694654e0aab3c3fd0fa3bedf1/users/96ac1ad045b4494980a2bfc74f1aa27f/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.948717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc4f15-4629-49eb-bb70-3530518b1483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffec7ef694654e0aab3c3fd0fa3bedf1, user_id=96ac1ad045b4494980a2bfc74f1aa27f, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:00.951411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc4f15-4629-49eb-bb70-3530518b1483 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:00.955997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbc822-a7d7-47be-833a-ab337e89c880 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:00.958015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0bfbc822-a7d7-47be-833a-ab337e89c880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:00.963660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1028/2047] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:00 2022] PUT /identity/v3/projects/ffec7ef694654e0aab3c3fd0fa3bedf1/users/96ac1ad045b4494980a2bfc74f1aa27f/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:00.974878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeba631-c599-4c0a-8832-90161466762a None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:00.975167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeba631-c599-4c0a-8832-90161466762a None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:00.975429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeba631-c599-4c0a-8832-90161466762a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:00.975637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbc822-a7d7-47be-833a-ab337e89c880 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.005589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-afeba631-c599-4c0a-8832-90161466762a None None] MFA Rules not processed for user `96ac1ad045b4494980a2bfc74f1aa27f`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:01.006986 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-afeba631-c599-4c0a-8832-90161466762a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.042924 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.043877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.044175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.044532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] PATH_INFO: `/v3/projects/de628dd84ca743d98bbae8b13fe8d4c4` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.046912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=de628dd84ca743d98bbae8b13fe8d4c4)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.048884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.092458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1029/2048] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:01.102149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de628dd84ca743d98bbae8b13fe8d4c4', 'request_id': 'req-941c959b-c5cb-4552-b26e-7fc4e0722a6c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.102781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeba631-c599-4c0a-8832-90161466762a None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.104190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-afeba631-c599-4c0a-8832-90161466762a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.111396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de628dd84ca743d98bbae8b13fe8d4c4', 'request_id': 'req-941c959b-c5cb-4552-b26e-7fc4e0722a6c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.118888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de628dd84ca743d98bbae8b13fe8d4c4', 'request_id': 'req-941c959b-c5cb-4552-b26e-7fc4e0722a6c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.131054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] The token cache is being invalidate because project de628dd84ca743d98bbae8b13fe8d4c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:01.131340 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.132014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-941c959b-c5cb-4552-b26e-7fc4e0722a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.133893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1020/2049] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:00 2022] DELETE /identity/v3/projects/de628dd84ca743d98bbae8b13fe8d4c4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:01.139324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e334c6-ef1b-4974-a3a3-501d4599bac6 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.139722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e334c6-ef1b-4974-a3a3-501d4599bac6 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.140058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e334c6-ef1b-4974-a3a3-501d4599bac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.164951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-35e334c6-ef1b-4974-a3a3-501d4599bac6 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:01.166301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-35e334c6-ef1b-4974-a3a3-501d4599bac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.192111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeba631-c599-4c0a-8832-90161466762a None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.217360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:01.243856 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1021/2050] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:01.258260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f404-cbfe-4def-aea8-39e275d0b906 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.258969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f404-cbfe-4def-aea8-39e275d0b906 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.259405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f404-cbfe-4def-aea8-39e275d0b906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.276220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041461fd-222a-442b-b115-a3e1ff461a1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.276929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041461fd-222a-442b-b115-a3e1ff461a1d service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.277215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041461fd-222a-442b-b115-a3e1ff461a1d service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.277514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041461fd-222a-442b-b115-a3e1ff461a1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.279368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-041461fd-222a-442b-b115-a3e1ff461a1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.283016 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-fb45f404-cbfe-4def-aea8-39e275d0b906 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:01.284312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fb45f404-cbfe-4def-aea8-39e275d0b906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.293014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1022/2051] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:01.297471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45f404-cbfe-4def-aea8-39e275d0b906 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.323254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45f404-cbfe-4def-aea8-39e275d0b906 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.329178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b77e1-8713-4882-ad23-ededacb30631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.329944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b77e1-8713-4882-ad23-ededacb30631 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.331171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b77e1-8713-4882-ad23-ededacb30631 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.331457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b77e1-8713-4882-ad23-ededacb30631 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.332247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b77e1-8713-4882-ad23-ededacb30631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:01.332597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b77e1-8713-4882-ad23-ededacb30631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.334276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b77e1-8713-4882-ad23-ededacb30631 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.344598 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:01.350704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1023/2052] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:56:01.355831 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b77e1-8713-4882-ad23-ededacb30631 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.368883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b77e1-8713-4882-ad23-ededacb30631 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.374513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041461fd-222a-442b-b115-a3e1ff461a1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.374699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89b467-1c5d-496e-b15f-9aa89b60da85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.375379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89b467-1c5d-496e-b15f-9aa89b60da85 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.375660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89b467-1c5d-496e-b15f-9aa89b60da85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.375936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89b467-1c5d-496e-b15f-9aa89b60da85 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.376095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041461fd-222a-442b-b115-a3e1ff461a1d service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.376663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89b467-1c5d-496e-b15f-9aa89b60da85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.378558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89b467-1c5d-496e-b15f-9aa89b60da85 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.387921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1030/2053] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:56:01.417619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1024/2054] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:01 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:01.421782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041461fd-222a-442b-b115-a3e1ff461a1d service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.433463 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041461fd-222a-442b-b115-a3e1ff461a1d service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.510431 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d0f3c-66c6-4c62-9f93-89889eb7bf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.511137 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d0f3c-66c6-4c62-9f93-89889eb7bf70 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.511427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d0f3c-66c6-4c62-9f93-89889eb7bf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.511705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d0f3c-66c6-4c62-9f93-89889eb7bf70 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.512519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d0f3c-66c6-4c62-9f93-89889eb7bf70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.514599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d0f3c-66c6-4c62-9f93-89889eb7bf70 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.538435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1031/2055] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:01 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:56:01.543272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89b467-1c5d-496e-b15f-9aa89b60da85 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.555081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89b467-1c5d-496e-b15f-9aa89b60da85 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.560961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314ca76-ef64-4ccd-9d78-4942a687f5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.562416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314ca76-ef64-4ccd-9d78-4942a687f5dd admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.562702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314ca76-ef64-4ccd-9d78-4942a687f5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.562978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314ca76-ef64-4ccd-9d78-4942a687f5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.563725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314ca76-ef64-4ccd-9d78-4942a687f5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.568136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314ca76-ef64-4ccd-9d78-4942a687f5dd admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.577899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1025/2056] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:01.582346 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d0f3c-66c6-4c62-9f93-89889eb7bf70 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.594190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d0f3c-66c6-4c62-9f93-89889eb7bf70 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.602114 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06182381-d722-4fca-ac26-ff21c67efed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.602796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06182381-d722-4fca-ac26-ff21c67efed0 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.603149 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06182381-d722-4fca-ac26-ff21c67efed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.603348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06182381-d722-4fca-ac26-ff21c67efed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.604088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06182381-d722-4fca-ac26-ff21c67efed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.605706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06182381-d722-4fca-ac26-ff21c67efed0 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.619220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1032/2057] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:01.621641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314ca76-ef64-4ccd-9d78-4942a687f5dd admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.621981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06182381-d722-4fca-ac26-ff21c67efed0 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.623501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b314ca76-ef64-4ccd-9d78-4942a687f5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.633451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06182381-d722-4fca-ac26-ff21c67efed0 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.638214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c37acb-6b27-408d-ba76-6aa50d1acbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.639102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c37acb-6b27-408d-ba76-6aa50d1acbc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.639387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c37acb-6b27-408d-ba76-6aa50d1acbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.639684 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c37acb-6b27-408d-ba76-6aa50d1acbc1 admin admin] PATH_INFO: `/v3/projects/13840d3a198645bb8569b34602f57bf5/users/d743e77d34aa493c8e244c8969b9dbe9/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.643830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314ca76-ef64-4ccd-9d78-4942a687f5dd admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.648791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5abf08-1af3-46a3-bf19-5251b00ba867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.649698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5abf08-1af3-46a3-bf19-5251b00ba867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.650109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5abf08-1af3-46a3-bf19-5251b00ba867 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.650392 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5abf08-1af3-46a3-bf19-5251b00ba867 admin admin] PATH_INFO: `/v3/users/1b9330b410f04d759124380d3c4f04bc` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.651632 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5abf08-1af3-46a3-bf19-5251b00ba867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9330b410f04d759124380d3c4f04bc)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.653599 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5abf08-1af3-46a3-bf19-5251b00ba867 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.655883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c37acb-6b27-408d-ba76-6aa50d1acbc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13840d3a198645bb8569b34602f57bf5, user_id=d743e77d34aa493c8e244c8969b9dbe9, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.658657 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c37acb-6b27-408d-ba76-6aa50d1acbc1 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.673867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1033/2058] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:01 2022] PUT /identity/v3/projects/13840d3a198645bb8569b34602f57bf5/users/d743e77d34aa493c8e244c8969b9dbe9/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:01.685757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e9980-be67-421b-beb9-3e020dd9745e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.686087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e9980-be67-421b-beb9-3e020dd9745e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.686413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e9980-be67-421b-beb9-3e020dd9745e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.710368 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de5abf08-1af3-46a3-bf19-5251b00ba867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9330b410f04d759124380d3c4f04bc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.719063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-132e9980-be67-421b-beb9-3e020dd9745e None None] MFA Rules not processed for user `d743e77d34aa493c8e244c8969b9dbe9`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:01.719319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de5abf08-1af3-46a3-bf19-5251b00ba867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9330b410f04d759124380d3c4f04bc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.720339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-132e9980-be67-421b-beb9-3e020dd9745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.729601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de5abf08-1af3-46a3-bf19-5251b00ba867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9330b410f04d759124380d3c4f04bc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.730370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:01.730721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de5abf08-1af3-46a3-bf19-5251b00ba867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9330b410f04d759124380d3c4f04bc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.746229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de5abf08-1af3-46a3-bf19-5251b00ba867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9330b410f04d759124380d3c4f04bc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:01.757738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1026/2059] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:01 2022] DELETE /identity/v3/users/1b9330b410f04d759124380d3c4f04bc => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:01.759323 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867659a-fa88-4c2c-8d3d-3d642a7fc762 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.759623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867659a-fa88-4c2c-8d3d-3d642a7fc762 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.759904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867659a-fa88-4c2c-8d3d-3d642a7fc762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.778560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4867659a-fa88-4c2c-8d3d-3d642a7fc762 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:01.780075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4867659a-fa88-4c2c-8d3d-3d642a7fc762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.798666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1034/2060] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:01.811448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e9980-be67-421b-beb9-3e020dd9745e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.813848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-132e9980-be67-421b-beb9-3e020dd9745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.861329 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1027/2061] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:01.875425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867659a-fa88-4c2c-8d3d-3d642a7fc762 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:01.876885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4867659a-fa88-4c2c-8d3d-3d642a7fc762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:01.881067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e9980-be67-421b-beb9-3e020dd9745e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.931799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:01.940087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183a4e7-20b5-41f2-859b-71243ba49768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.940812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183a4e7-20b5-41f2-859b-71243ba49768 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.941095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183a4e7-20b5-41f2-859b-71243ba49768 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.941410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183a4e7-20b5-41f2-859b-71243ba49768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.949589 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867659a-fa88-4c2c-8d3d-3d642a7fc762 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:01.953463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e8492-300a-4349-9bc8-013349e42330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:01.954174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e8492-300a-4349-9bc8-013349e42330 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:01.954464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e8492-300a-4349-9bc8-013349e42330 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:01.954737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e8492-300a-4349-9bc8-013349e42330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:01.964583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e8492-300a-4349-9bc8-013349e42330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:01.966029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e8492-300a-4349-9bc8-013349e42330 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:01.977551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1028/2062] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:56:02.007209 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183a4e7-20b5-41f2-859b-71243ba49768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:02.008690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183a4e7-20b5-41f2-859b-71243ba49768 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:02.018279 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1035/2063] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:56:02.313344 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e8492-300a-4349-9bc8-013349e42330 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:02.315396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b42e8492-300a-4349-9bc8-013349e42330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:02.333346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e8492-300a-4349-9bc8-013349e42330 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:02.340540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:02.341426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:02.341748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:02.342035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] PATH_INFO: `/v3/projects/63c8f7374da74cd8ae2d66e751a75484` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:02.343857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c8f7374da74cd8ae2d66e751a75484)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:02.345833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:02.434033 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c8f7374da74cd8ae2d66e751a75484', 'request_id': 'req-9d0d4819-7d13-425e-bd19-9dda1c42a653'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:02.443267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c8f7374da74cd8ae2d66e751a75484', 'request_id': 'req-9d0d4819-7d13-425e-bd19-9dda1c42a653'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:02.457509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c8f7374da74cd8ae2d66e751a75484', 'request_id': 'req-9d0d4819-7d13-425e-bd19-9dda1c42a653'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:02.472762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] The token cache is being invalidate because project 63c8f7374da74cd8ae2d66e751a75484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:02.473491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:02.474162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:02.475750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1029/2064] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:02 2022] DELETE /identity/v3/projects/63c8f7374da74cd8ae2d66e751a75484 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:02.480593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183a4e7-20b5-41f2-859b-71243ba49768 service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:02.500830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183a4e7-20b5-41f2-859b-71243ba49768 service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:02.526064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:02.590976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485200ba-3d74-4c34-8c2f-b9a286146e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:02.592354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485200ba-3d74-4c34-8c2f-b9a286146e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:02.593817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485200ba-3d74-4c34-8c2f-b9a286146e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:02.594121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485200ba-3d74-4c34-8c2f-b9a286146e05 admin admin] PATH_INFO: `/v3/users/96ac1ad045b4494980a2bfc74f1aa27f` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:02.595391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485200ba-3d74-4c34-8c2f-b9a286146e05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96ac1ad045b4494980a2bfc74f1aa27f)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:02.597528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485200ba-3d74-4c34-8c2f-b9a286146e05 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:02.653218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-485200ba-3d74-4c34-8c2f-b9a286146e05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96ac1ad045b4494980a2bfc74f1aa27f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:02.662269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-485200ba-3d74-4c34-8c2f-b9a286146e05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96ac1ad045b4494980a2bfc74f1aa27f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:02.662994 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-485200ba-3d74-4c34-8c2f-b9a286146e05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96ac1ad045b4494980a2bfc74f1aa27f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:02.675171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-485200ba-3d74-4c34-8c2f-b9a286146e05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96ac1ad045b4494980a2bfc74f1aa27f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:02.682085 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-485200ba-3d74-4c34-8c2f-b9a286146e05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96ac1ad045b4494980a2bfc74f1aa27f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:02.688579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1036/2065] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:02 2022] DELETE /identity/v3/users/96ac1ad045b4494980a2bfc74f1aa27f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:02.978592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:02.980124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:02.993935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d4819-7d13-425e-bd19-9dda1c42a653 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.054022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.054894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.055178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.055455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] PATH_INFO: `/v3/projects/ffec7ef694654e0aab3c3fd0fa3bedf1` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.056688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffec7ef694654e0aab3c3fd0fa3bedf1)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.058601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.106479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffec7ef694654e0aab3c3fd0fa3bedf1', 'request_id': 'req-095c98d0-c066-43a9-b079-3025f43b5a94'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.114522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffec7ef694654e0aab3c3fd0fa3bedf1', 'request_id': 'req-095c98d0-c066-43a9-b079-3025f43b5a94'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.120883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffec7ef694654e0aab3c3fd0fa3bedf1', 'request_id': 'req-095c98d0-c066-43a9-b079-3025f43b5a94'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.125788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:03.135170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] The token cache is being invalidate because project ffec7ef694654e0aab3c3fd0fa3bedf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:03.135484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.136442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-095c98d0-c066-43a9-b079-3025f43b5a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:03.138193 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1030/2066] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:02 2022] DELETE /identity/v3/projects/ffec7ef694654e0aab3c3fd0fa3bedf1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:03.143313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834851d3-9a30-4692-8efa-0323364b4c6c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.143659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834851d3-9a30-4692-8efa-0323364b4c6c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.143964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834851d3-9a30-4692-8efa-0323364b4c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.162366 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-834851d3-9a30-4692-8efa-0323364b4c6c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:03.163590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-834851d3-9a30-4692-8efa-0323364b4c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:03.226617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1037/2067] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:03.236234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb822046-13c7-4fe3-9174-af8c0f16d283 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.236559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb822046-13c7-4fe3-9174-af8c0f16d283 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.236843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb822046-13c7-4fe3-9174-af8c0f16d283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.256367 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-eb822046-13c7-4fe3-9174-af8c0f16d283 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:03.257563 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-eb822046-13c7-4fe3-9174-af8c0f16d283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:03.265702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1031/2068] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:03.269988 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834851d3-9a30-4692-8efa-0323364b4c6c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.279849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834851d3-9a30-4692-8efa-0323364b4c6c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.284533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52719ef7-2e8a-47f7-b156-fce2570b2729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.285172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719ef7-2e8a-47f7-b156-fce2570b2729 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.285459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719ef7-2e8a-47f7-b156-fce2570b2729 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.285740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719ef7-2e8a-47f7-b156-fce2570b2729 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.286471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52719ef7-2e8a-47f7-b156-fce2570b2729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:03.286782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52719ef7-2e8a-47f7-b156-fce2570b2729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.288937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52719ef7-2e8a-47f7-b156-fce2570b2729 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.301115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1038/2069] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:56:03.306185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb822046-13c7-4fe3-9174-af8c0f16d283 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.316827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb822046-13c7-4fe3-9174-af8c0f16d283 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.322307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea07ac1-2df8-4a65-9944-a713d1b10e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.322965 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea07ac1-2df8-4a65-9944-a713d1b10e25 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.323250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea07ac1-2df8-4a65-9944-a713d1b10e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.323533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea07ac1-2df8-4a65-9944-a713d1b10e25 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.324226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea07ac1-2df8-4a65-9944-a713d1b10e25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.326166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea07ac1-2df8-4a65-9944-a713d1b10e25 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.354076 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1032/2070] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:03 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:03.358282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52719ef7-2e8a-47f7-b156-fce2570b2729 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.369806 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52719ef7-2e8a-47f7-b156-fce2570b2729 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.397378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:03.435674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12412c-2f40-4320-a701-b75c01663ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.436918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12412c-2f40-4320-a701-b75c01663ffe admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.436918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12412c-2f40-4320-a701-b75c01663ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.437107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12412c-2f40-4320-a701-b75c01663ffe admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.437670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12412c-2f40-4320-a701-b75c01663ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.439951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12412c-2f40-4320-a701-b75c01663ffe admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.467450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1039/2071] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:03 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:03.469396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea07ac1-2df8-4a65-9944-a713d1b10e25 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.486669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea07ac1-2df8-4a65-9944-a713d1b10e25 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.493550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8d97c-27b9-4c11-a3a4-63db781812d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.494257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8d97c-27b9-4c11-a3a4-63db781812d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.494542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8d97c-27b9-4c11-a3a4-63db781812d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.494824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8d97c-27b9-4c11-a3a4-63db781812d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.495559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8d97c-27b9-4c11-a3a4-63db781812d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.497181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8d97c-27b9-4c11-a3a4-63db781812d6 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.508589 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1033/2072] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:03.512921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12412c-2f40-4320-a701-b75c01663ffe admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.524003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12412c-2f40-4320-a701-b75c01663ffe admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.528701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c2eee-0942-4666-9a8d-f298e7d09bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.529370 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2eee-0942-4666-9a8d-f298e7d09bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.529655 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2eee-0942-4666-9a8d-f298e7d09bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.529934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2eee-0942-4666-9a8d-f298e7d09bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.531790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c2eee-0942-4666-9a8d-f298e7d09bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.532990 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c2eee-0942-4666-9a8d-f298e7d09bd0 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.547541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1040/2073] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:03.548731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8d97c-27b9-4c11-a3a4-63db781812d6 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.559730 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8d97c-27b9-4c11-a3a4-63db781812d6 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.564919 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb3173-d440-412d-be9c-1526f30a309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.565818 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb3173-d440-412d-be9c-1526f30a309b admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.566100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb3173-d440-412d-be9c-1526f30a309b admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.566376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb3173-d440-412d-be9c-1526f30a309b admin admin] PATH_INFO: `/v3/projects/252072133e6d4600a6fa490d46924d56/users/0e373b781e444627a66612413a74edcb/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.584218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb3173-d440-412d-be9c-1526f30a309b admin admin] RBAC: Authorizing `identity:create_grant(project_id=252072133e6d4600a6fa490d46924d56, user_id=0e373b781e444627a66612413a74edcb, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.586826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb3173-d440-412d-be9c-1526f30a309b admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.606598 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1034/2074] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:03 2022] PUT /identity/v3/projects/252072133e6d4600a6fa490d46924d56/users/0e373b781e444627a66612413a74edcb/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:03.621664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0668256-aeb2-48fb-b672-f998c04b8e1d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.621963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0668256-aeb2-48fb-b672-f998c04b8e1d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.622844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0668256-aeb2-48fb-b672-f998c04b8e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.656781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c0668256-aeb2-48fb-b672-f998c04b8e1d None None] MFA Rules not processed for user `0e373b781e444627a66612413a74edcb`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:03.658024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c0668256-aeb2-48fb-b672-f998c04b8e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:03.736702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1041/2075] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:03.741242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb3173-d440-412d-be9c-1526f30a309b admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.806519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb3173-d440-412d-be9c-1526f30a309b admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.811582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31ec9b-9616-4808-9b60-01be1ee4bad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.812398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31ec9b-9616-4808-9b60-01be1ee4bad4 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.812875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31ec9b-9616-4808-9b60-01be1ee4bad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.812990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31ec9b-9616-4808-9b60-01be1ee4bad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.813652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31ec9b-9616-4808-9b60-01be1ee4bad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.815839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31ec9b-9616-4808-9b60-01be1ee4bad4 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.841914 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1035/2076] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:03 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:03.846734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0668256-aeb2-48fb-b672-f998c04b8e1d None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.857798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0668256-aeb2-48fb-b672-f998c04b8e1d None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.918460 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039fa154-493a-4aef-a14c-5308453f46ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.919142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fa154-493a-4aef-a14c-5308453f46ee admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.919427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fa154-493a-4aef-a14c-5308453f46ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.919703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fa154-493a-4aef-a14c-5308453f46ee admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.920436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039fa154-493a-4aef-a14c-5308453f46ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.922409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039fa154-493a-4aef-a14c-5308453f46ee admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.946069 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1042/2077] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:03 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:03.949802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31ec9b-9616-4808-9b60-01be1ee4bad4 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.961104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31ec9b-9616-4808-9b60-01be1ee4bad4 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:03.965809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb641e7c-4378-40ef-9af5-cddd327b652d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:03.966473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb641e7c-4378-40ef-9af5-cddd327b652d admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:03.966849 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb641e7c-4378-40ef-9af5-cddd327b652d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:03.967027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb641e7c-4378-40ef-9af5-cddd327b652d admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:03.967792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb641e7c-4378-40ef-9af5-cddd327b652d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:03.969936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb641e7c-4378-40ef-9af5-cddd327b652d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:03.982229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1036/2078] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:03.986069 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039fa154-493a-4aef-a14c-5308453f46ee admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:03.996961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039fa154-493a-4aef-a14c-5308453f46ee admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.002119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6bd6c-b078-4302-83e7-01b9cc0a80c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.002843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6bd6c-b078-4302-83e7-01b9cc0a80c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.003129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6bd6c-b078-4302-83e7-01b9cc0a80c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.003489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6bd6c-b078-4302-83e7-01b9cc0a80c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.004304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6bd6c-b078-4302-83e7-01b9cc0a80c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.005981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6bd6c-b078-4302-83e7-01b9cc0a80c5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.018300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1043/2079] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:04.022357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb641e7c-4378-40ef-9af5-cddd327b652d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.033471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb641e7c-4378-40ef-9af5-cddd327b652d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.038370 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef374ec4-7f69-4a4a-a16f-62b5c292a25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.039333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef374ec4-7f69-4a4a-a16f-62b5c292a25d admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.039618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef374ec4-7f69-4a4a-a16f-62b5c292a25d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.039896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef374ec4-7f69-4a4a-a16f-62b5c292a25d admin admin] PATH_INFO: `/v3/projects/1c5543da1ea648f7a07da35da4dfd371/users/ed60f11059404a749680e5a3390a667b/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.056257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef374ec4-7f69-4a4a-a16f-62b5c292a25d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5543da1ea648f7a07da35da4dfd371, user_id=ed60f11059404a749680e5a3390a667b, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.059242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef374ec4-7f69-4a4a-a16f-62b5c292a25d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.074199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1037/2080] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:04 2022] PUT /identity/v3/projects/1c5543da1ea648f7a07da35da4dfd371/users/ed60f11059404a749680e5a3390a667b/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:04.086239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac5b4d-25ed-4ca5-88ef-bfcc08463183 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.086544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac5b4d-25ed-4ca5-88ef-bfcc08463183 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.086860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac5b4d-25ed-4ca5-88ef-bfcc08463183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.114754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:04.123074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-5bac5b4d-25ed-4ca5-88ef-bfcc08463183 None None] MFA Rules not processed for user `ed60f11059404a749680e5a3390a667b`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:04.124369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-5bac5b4d-25ed-4ca5-88ef-bfcc08463183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:04.194749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1044/2081] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:04.199391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef374ec4-7f69-4a4a-a16f-62b5c292a25d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.246855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:04.271624 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef374ec4-7f69-4a4a-a16f-62b5c292a25d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.276477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635c20b-aa75-4223-8738-792b9a17e930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.277132 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635c20b-aa75-4223-8738-792b9a17e930 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.277413 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635c20b-aa75-4223-8738-792b9a17e930 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.277685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635c20b-aa75-4223-8738-792b9a17e930 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.278362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635c20b-aa75-4223-8738-792b9a17e930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.280249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635c20b-aa75-4223-8738-792b9a17e930 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.307874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1038/2082] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:04 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:04.310527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac5b4d-25ed-4ca5-88ef-bfcc08463183 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.322022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac5b4d-25ed-4ca5-88ef-bfcc08463183 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.378576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b6d48-0289-4351-959c-74a1b00d4742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.379253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b6d48-0289-4351-959c-74a1b00d4742 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.379541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b6d48-0289-4351-959c-74a1b00d4742 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.379822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b6d48-0289-4351-959c-74a1b00d4742 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.380880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b6d48-0289-4351-959c-74a1b00d4742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.382488 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b6d48-0289-4351-959c-74a1b00d4742 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.405410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1045/2083] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:04 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:04.409625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635c20b-aa75-4223-8738-792b9a17e930 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.424359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635c20b-aa75-4223-8738-792b9a17e930 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.429466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd630984-9d54-4479-979f-026937d19a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.430210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd630984-9d54-4479-979f-026937d19a5e admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.430532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd630984-9d54-4479-979f-026937d19a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.430849 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd630984-9d54-4479-979f-026937d19a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.431706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd630984-9d54-4479-979f-026937d19a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.433430 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd630984-9d54-4479-979f-026937d19a5e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.448201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1039/2084] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:04.453228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b6d48-0289-4351-959c-74a1b00d4742 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.464111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b6d48-0289-4351-959c-74a1b00d4742 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.469373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4132d-c6f3-431a-aa81-28edd57680be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.470302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4132d-c6f3-431a-aa81-28edd57680be admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.470590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4132d-c6f3-431a-aa81-28edd57680be admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.470869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4132d-c6f3-431a-aa81-28edd57680be admin admin] PATH_INFO: `/v3/projects/87fbb253c94647b79decffa5ef06b11c/users/c574e57899474a93a7290906ec2d1d99/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.487010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4132d-c6f3-431a-aa81-28edd57680be admin admin] RBAC: Authorizing `identity:create_grant(project_id=87fbb253c94647b79decffa5ef06b11c, user_id=c574e57899474a93a7290906ec2d1d99, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.489782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4132d-c6f3-431a-aa81-28edd57680be admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.505881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1046/2085] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:04 2022] PUT /identity/v3/projects/87fbb253c94647b79decffa5ef06b11c/users/c574e57899474a93a7290906ec2d1d99/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:04.509987 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd630984-9d54-4479-979f-026937d19a5e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.521463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd630984-9d54-4479-979f-026937d19a5e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.589792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcc99c-99b4-4af8-9213-554f107cbcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.590506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc99c-99b4-4af8-9213-554f107cbcde admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.590785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc99c-99b4-4af8-9213-554f107cbcde admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.591061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc99c-99b4-4af8-9213-554f107cbcde admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.591974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcc99c-99b4-4af8-9213-554f107cbcde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.593651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcc99c-99b4-4af8-9213-554f107cbcde admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.604079 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1040/2086] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:04.608269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4132d-c6f3-431a-aa81-28edd57680be admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.619991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4132d-c6f3-431a-aa81-28edd57680be admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.625708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bb98e-f1c0-4463-84b3-ad2874705dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.626906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb98e-f1c0-4463-84b3-ad2874705dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.627268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb98e-f1c0-4463-84b3-ad2874705dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.627625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb98e-f1c0-4463-84b3-ad2874705dd0 admin admin] PATH_INFO: `/v3/projects/87fbb253c94647b79decffa5ef06b11c/users/c574e57899474a93a7290906ec2d1d99/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.630453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb98e-f1c0-4463-84b3-ad2874705dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87fbb253c94647b79decffa5ef06b11c, user_id=c574e57899474a93a7290906ec2d1d99, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.633058 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb98e-f1c0-4463-84b3-ad2874705dd0 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.644728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1047/2087] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:04 2022] PUT /identity/v3/projects/87fbb253c94647b79decffa5ef06b11c/users/c574e57899474a93a7290906ec2d1d99/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:04.654992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fa79e-dad3-4e89-940b-fa1f0efaba74 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.655397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fa79e-dad3-4e89-940b-fa1f0efaba74 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.655686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fa79e-dad3-4e89-940b-fa1f0efaba74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.687439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-f83fa79e-dad3-4e89-940b-fa1f0efaba74 None None] MFA Rules not processed for user `c574e57899474a93a7290906ec2d1d99`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:04.688804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f83fa79e-dad3-4e89-940b-fa1f0efaba74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:04.763501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1041/2088] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:04.775065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bb98e-f1c0-4463-84b3-ad2874705dd0 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.777161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-840bb98e-f1c0-4463-84b3-ad2874705dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:04.841722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bb98e-f1c0-4463-84b3-ad2874705dd0 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.845506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96d806-a05d-4fc1-88af-2fd1681e1a36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.846155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96d806-a05d-4fc1-88af-2fd1681e1a36 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.846447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96d806-a05d-4fc1-88af-2fd1681e1a36 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.846725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96d806-a05d-4fc1-88af-2fd1681e1a36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.856813 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96d806-a05d-4fc1-88af-2fd1681e1a36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:04.858253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96d806-a05d-4fc1-88af-2fd1681e1a36 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:04.863104 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:04.871845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1048/2089] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:04.914708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fa79e-dad3-4e89-940b-fa1f0efaba74 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:04.931522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fa79e-dad3-4e89-940b-fa1f0efaba74 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:04.935532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bccc6-ce4e-4992-a6ad-8e4bb1d6b2cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:04.936199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bccc6-ce4e-4992-a6ad-8e4bb1d6b2cc service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:04.936603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bccc6-ce4e-4992-a6ad-8e4bb1d6b2cc service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:04.936866 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bccc6-ce4e-4992-a6ad-8e4bb1d6b2cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:04.978031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:05.005786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bccc6-ce4e-4992-a6ad-8e4bb1d6b2cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:05.007227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bccc6-ce4e-4992-a6ad-8e4bb1d6b2cc service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:05.016954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1042/2090] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:14.872755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:15.012290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=12 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:22.708493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96d806-a05d-4fc1-88af-2fd1681e1a36 service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:22.782752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96d806-a05d-4fc1-88af-2fd1681e1a36 service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:22.789022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:22.790170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:22.790666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:22.791042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] PATH_INFO: `/v3/users/0e373b781e444627a66612413a74edcb` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:22.792784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e373b781e444627a66612413a74edcb)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:22.795286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:22.884931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e373b781e444627a66612413a74edcb'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:22.901527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e373b781e444627a66612413a74edcb'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:22.901527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e373b781e444627a66612413a74edcb'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:22.917747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e373b781e444627a66612413a74edcb'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:22.927038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e373b781e444627a66612413a74edcb'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:22.933542 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1049/2091] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:22 2022] DELETE /identity/v3/users/0e373b781e444627a66612413a74edcb => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:22.938952 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa810289-f3a1-4465-b1bb-0461eaaf3756 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:22.939676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa810289-f3a1-4465-b1bb-0461eaaf3756 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:22.940100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa810289-f3a1-4465-b1bb-0461eaaf3756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:22.959753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-fa810289-f3a1-4465-b1bb-0461eaaf3756 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:22.961930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fa810289-f3a1-4465-b1bb-0461eaaf3756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:23.057310 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1043/2092] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:23.072634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:23.072878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:23.158681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51987ddf-1413-462b-a2e8-1d79472f294f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:23.169275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40986f86-4f0a-430e-bde3-7151d4a500d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:23.170052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986f86-4f0a-430e-bde3-7151d4a500d7 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:23.170052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986f86-4f0a-430e-bde3-7151d4a500d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:23.172479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40986f86-4f0a-430e-bde3-7151d4a500d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:23.190985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40986f86-4f0a-430e-bde3-7151d4a500d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:23.193239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40986f86-4f0a-430e-bde3-7151d4a500d7 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:23.215674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1050/2093] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:23.249755 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa810289-f3a1-4465-b1bb-0461eaaf3756 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:23.251347 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fa810289-f3a1-4465-b1bb-0461eaaf3756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:23.270257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa810289-f3a1-4465-b1bb-0461eaaf3756 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:23.276059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:23.277025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:23.277306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:23.277635 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] PATH_INFO: `/v3/users/d743e77d34aa493c8e244c8969b9dbe9` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:23.278930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d743e77d34aa493c8e244c8969b9dbe9)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:23.281009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:23.355991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd743e77d34aa493c8e244c8969b9dbe9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.369677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd743e77d34aa493c8e244c8969b9dbe9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.378717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd743e77d34aa493c8e244c8969b9dbe9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.379448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd743e77d34aa493c8e244c8969b9dbe9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.394225 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd743e77d34aa493c8e244c8969b9dbe9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.401192 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1044/2094] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:23 2022] DELETE /identity/v3/users/d743e77d34aa493c8e244c8969b9dbe9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:23.405711 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68cf2-cb95-4720-aae7-e7f4ef2a0e12 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:23.406001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68cf2-cb95-4720-aae7-e7f4ef2a0e12 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:23.406336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68cf2-cb95-4720-aae7-e7f4ef2a0e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:23.431854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-dbe68cf2-cb95-4720-aae7-e7f4ef2a0e12 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:23.433239 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-dbe68cf2-cb95-4720-aae7-e7f4ef2a0e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:23.471904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:23.530641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1051/2095] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:23.548680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:23.550170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:23.582004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe68cf2-cb95-4720-aae7-e7f4ef2a0e12 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:23.584184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-dbe68cf2-cb95-4720-aae7-e7f4ef2a0e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:23.600942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe68cf2-cb95-4720-aae7-e7f4ef2a0e12 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:23.613874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:23.616752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:23.617187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:23.617295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] PATH_INFO: `/v3/projects/252072133e6d4600a6fa490d46924d56` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:23.619123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] RBAC: Authorizing `identity:delete_project(project_id=252072133e6d4600a6fa490d46924d56)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:23.622907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:23.631732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8e5ae-5b13-4bee-a8bc-1dfb890f1cfd admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:23.637588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070faa51-6a50-4b42-b741-bee83e90fee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:23.638153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070faa51-6a50-4b42-b741-bee83e90fee0 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:23.638444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070faa51-6a50-4b42-b741-bee83e90fee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:23.638788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070faa51-6a50-4b42-b741-bee83e90fee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:23.655627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070faa51-6a50-4b42-b741-bee83e90fee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:23.659093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070faa51-6a50-4b42-b741-bee83e90fee0 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:23.671869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1045/2096] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:23.691401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252072133e6d4600a6fa490d46924d56', 'request_id': 'req-a6770677-1341-4152-bd6c-09efc226a87a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.700202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252072133e6d4600a6fa490d46924d56', 'request_id': 'req-a6770677-1341-4152-bd6c-09efc226a87a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.707876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252072133e6d4600a6fa490d46924d56', 'request_id': 'req-a6770677-1341-4152-bd6c-09efc226a87a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.721568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] The token cache is being invalidate because project 252072133e6d4600a6fa490d46924d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:23.722203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.722672 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.725474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1052/2097] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:23 2022] DELETE /identity/v3/projects/252072133e6d4600a6fa490d46924d56 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:23.729533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070faa51-6a50-4b42-b741-bee83e90fee0 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:23.742335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070faa51-6a50-4b42-b741-bee83e90fee0 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:23.762424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:23.845360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf683826-89f2-4ef4-a7ae-ef7cd21c3bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:23.846565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683826-89f2-4ef4-a7ae-ef7cd21c3bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:23.847025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683826-89f2-4ef4-a7ae-ef7cd21c3bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:23.847525 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683826-89f2-4ef4-a7ae-ef7cd21c3bd2 admin admin] PATH_INFO: `/v3/users/ed60f11059404a749680e5a3390a667b` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:23.849228 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf683826-89f2-4ef4-a7ae-ef7cd21c3bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed60f11059404a749680e5a3390a667b)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:23.851426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf683826-89f2-4ef4-a7ae-ef7cd21c3bd2 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:23.919623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cf683826-89f2-4ef4-a7ae-ef7cd21c3bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed60f11059404a749680e5a3390a667b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.933071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cf683826-89f2-4ef4-a7ae-ef7cd21c3bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed60f11059404a749680e5a3390a667b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.941386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cf683826-89f2-4ef4-a7ae-ef7cd21c3bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed60f11059404a749680e5a3390a667b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.942381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cf683826-89f2-4ef4-a7ae-ef7cd21c3bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed60f11059404a749680e5a3390a667b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.956085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cf683826-89f2-4ef4-a7ae-ef7cd21c3bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed60f11059404a749680e5a3390a667b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:23.962987 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1046/2098] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:23 2022] DELETE /identity/v3/users/ed60f11059404a749680e5a3390a667b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:24.060538 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:24.063088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:24.079879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6770677-1341-4152-bd6c-09efc226a87a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:24.161060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:24.162513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.162799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.163075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] PATH_INFO: `/v3/projects/13840d3a198645bb8569b34602f57bf5` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.164501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=13840d3a198645bb8569b34602f57bf5)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:24.166641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:24.245931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:24.250365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13840d3a198645bb8569b34602f57bf5', 'request_id': 'req-d2d34681-10c5-4d6f-bc2b-265d66432d5b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.262067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13840d3a198645bb8569b34602f57bf5', 'request_id': 'req-d2d34681-10c5-4d6f-bc2b-265d66432d5b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.272900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13840d3a198645bb8569b34602f57bf5', 'request_id': 'req-d2d34681-10c5-4d6f-bc2b-265d66432d5b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.297673 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] The token cache is being invalidate because project 13840d3a198645bb8569b34602f57bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:24.298066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.298952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.300815 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1053/2099] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:24 2022] DELETE /identity/v3/projects/13840d3a198645bb8569b34602f57bf5 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:24.306110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2a3ed-ff55-4fe8-81ce-2670113e9ba4 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.306403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2a3ed-ff55-4fe8-81ce-2670113e9ba4 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.306790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2a3ed-ff55-4fe8-81ce-2670113e9ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.338712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-6bd2a3ed-ff55-4fe8-81ce-2670113e9ba4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:24.340053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6bd2a3ed-ff55-4fe8-81ce-2670113e9ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:24.379734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:24.381212 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:24.434050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1047/2100] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:24.444098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a60be-e8a7-4b18-ad9d-a7292abc2613 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.444430 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a60be-e8a7-4b18-ad9d-a7292abc2613 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.444749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a60be-e8a7-4b18-ad9d-a7292abc2613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.467189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-249a60be-e8a7-4b18-ad9d-a7292abc2613 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:24.468947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-249a60be-e8a7-4b18-ad9d-a7292abc2613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:24.470575 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d34681-10c5-4d6f-bc2b-265d66432d5b admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:24.475997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:24.476899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.477188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.477469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] PATH_INFO: `/v3/projects/1c5543da1ea648f7a07da35da4dfd371` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.477784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1048/2101] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:24.478663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c5543da1ea648f7a07da35da4dfd371)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:24.480635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:24.482047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a60be-e8a7-4b18-ad9d-a7292abc2613 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:24.491283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a60be-e8a7-4b18-ad9d-a7292abc2613 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:24.496394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49dbc82-d78c-4eee-bbfa-b51f1af805a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:24.497057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dbc82-d78c-4eee-bbfa-b51f1af805a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.497362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dbc82-d78c-4eee-bbfa-b51f1af805a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.497637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49dbc82-d78c-4eee-bbfa-b51f1af805a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.498355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49dbc82-d78c-4eee-bbfa-b51f1af805a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:24.499150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49dbc82-d78c-4eee-bbfa-b51f1af805a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:24.500861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49dbc82-d78c-4eee-bbfa-b51f1af805a8 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:24.517138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1049/2102] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:56:24.518146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49dbc82-d78c-4eee-bbfa-b51f1af805a8 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:24.528170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49dbc82-d78c-4eee-bbfa-b51f1af805a8 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:24.533515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c5543da1ea648f7a07da35da4dfd371', 'request_id': 'req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.542868 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:24.543109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c5543da1ea648f7a07da35da4dfd371', 'request_id': 'req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.549640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c5543da1ea648f7a07da35da4dfd371', 'request_id': 'req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.561886 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] The token cache is being invalidate because project 1c5543da1ea648f7a07da35da4dfd371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:24.562178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.562866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.564768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1054/2103] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:24 2022] DELETE /identity/v3/projects/1c5543da1ea648f7a07da35da4dfd371 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:24.570139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:24.571527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:24.599023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e130-f19c-4786-b2bd-5a32d4c2b233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:24.599845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e130-f19c-4786-b2bd-5a32d4c2b233 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.600224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e130-f19c-4786-b2bd-5a32d4c2b233 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.600601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e130-f19c-4786-b2bd-5a32d4c2b233 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.601306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e130-f19c-4786-b2bd-5a32d4c2b233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:24.603348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e130-f19c-4786-b2bd-5a32d4c2b233 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:24.632454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1050/2104] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:24 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:56:24.636353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e130-f19c-4786-b2bd-5a32d4c2b233 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:24.642380 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-9104e130-f19c-4786-b2bd-5a32d4c2b233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:24.647872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad80097-a2c4-48ab-b6fb-7a8091d25b1b admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:24.660300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:24.714614 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:24.715511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.715801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.716082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] PATH_INFO: `/v3/users/c574e57899474a93a7290906ec2d1d99` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.717373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c574e57899474a93a7290906ec2d1d99)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:24.718249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e130-f19c-4786-b2bd-5a32d4c2b233 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:24.719311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:24.723111 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21d56b-daab-4bb6-9abc-f223773a6daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:24.723787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21d56b-daab-4bb6-9abc-f223773a6daa admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.724064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21d56b-daab-4bb6-9abc-f223773a6daa admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.724354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21d56b-daab-4bb6-9abc-f223773a6daa admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.725804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21d56b-daab-4bb6-9abc-f223773a6daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:24.727007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21d56b-daab-4bb6-9abc-f223773a6daa admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:24.753743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1051/2105] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:24 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:56:24.758232 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21d56b-daab-4bb6-9abc-f223773a6daa admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:24.770783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21d56b-daab-4bb6-9abc-f223773a6daa admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:24.775822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c574e57899474a93a7290906ec2d1d99'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.776294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940476f5-51cb-4cbc-9096-163183562b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:24.776955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940476f5-51cb-4cbc-9096-163183562b44 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.777237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940476f5-51cb-4cbc-9096-163183562b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.777514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940476f5-51cb-4cbc-9096-163183562b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.778242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940476f5-51cb-4cbc-9096-163183562b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:24.779886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940476f5-51cb-4cbc-9096-163183562b44 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:24.786892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c574e57899474a93a7290906ec2d1d99'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.788163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c574e57899474a93a7290906ec2d1d99'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.792398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1052/2106] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:24.796850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940476f5-51cb-4cbc-9096-163183562b44 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:24.798610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-940476f5-51cb-4cbc-9096-163183562b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:24.803017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c574e57899474a93a7290906ec2d1d99'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.811882 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940476f5-51cb-4cbc-9096-163183562b44 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:24.812163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c574e57899474a93a7290906ec2d1d99'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:24.817682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a619c-6843-492f-883f-701cecb840a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:24.818524 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a619c-6843-492f-883f-701cecb840a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.818819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a619c-6843-492f-883f-701cecb840a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.819099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a619c-6843-492f-883f-701cecb840a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.819987 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a619c-6843-492f-883f-701cecb840a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:24.821683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a619c-6843-492f-883f-701cecb840a4 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:24.825493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1055/2107] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:24 2022] DELETE /identity/v3/users/c574e57899474a93a7290906ec2d1d99 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:24.833243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1053/2108] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:24.837430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:24.915590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea04ae-1bb5-4b1d-8d6b-25db2cb761cf admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:24.921048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607440e-3cce-433a-bc8d-aa4165b31b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:24.921967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607440e-3cce-433a-bc8d-aa4165b31b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.922236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607440e-3cce-433a-bc8d-aa4165b31b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.922515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607440e-3cce-433a-bc8d-aa4165b31b8c admin admin] PATH_INFO: `/v3/projects/64e3949ba6b3406192340a338383947c/users/4b7276b3a0914806881c04495ed1ad6b/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.940357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607440e-3cce-433a-bc8d-aa4165b31b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e3949ba6b3406192340a338383947c, user_id=4b7276b3a0914806881c04495ed1ad6b, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:24.943108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607440e-3cce-433a-bc8d-aa4165b31b8c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:24.957667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1056/2109] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:24 2022] PUT /identity/v3/projects/64e3949ba6b3406192340a338383947c/users/4b7276b3a0914806881c04495ed1ad6b/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:24.968044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6aae9-caa3-4899-9e1a-aee0f1d4070b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:24.968565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6aae9-caa3-4899-9e1a-aee0f1d4070b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:24.968644 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6aae9-caa3-4899-9e1a-aee0f1d4070b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:24.999417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-dbd6aae9-caa3-4899-9e1a-aee0f1d4070b None None] MFA Rules not processed for user `4b7276b3a0914806881c04495ed1ad6b`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:25.000670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dbd6aae9-caa3-4899-9e1a-aee0f1d4070b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:25.066791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:25.071566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1054/2110] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:25.083360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607440e-3cce-433a-bc8d-aa4165b31b8c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:25.085410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c607440e-3cce-433a-bc8d-aa4165b31b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:25.105890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:25.127721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6aae9-caa3-4899-9e1a-aee0f1d4070b None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:25.130105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dbd6aae9-caa3-4899-9e1a-aee0f1d4070b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:25.155416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607440e-3cce-433a-bc8d-aa4165b31b8c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:25.164101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06a157-3765-4d36-a8dd-91e84a3e54a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:25.164101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06a157-3765-4d36-a8dd-91e84a3e54a3 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:25.164101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06a157-3765-4d36-a8dd-91e84a3e54a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:25.164101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06a157-3765-4d36-a8dd-91e84a3e54a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:25.176605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06a157-3765-4d36-a8dd-91e84a3e54a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:25.178318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06a157-3765-4d36-a8dd-91e84a3e54a3 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:25.188102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1057/2111] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:25.235199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6aae9-caa3-4899-9e1a-aee0f1d4070b None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:25.239923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:25.240843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:25.241125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:25.241400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] PATH_INFO: `/v3/projects/87fbb253c94647b79decffa5ef06b11c` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:25.242718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87fbb253c94647b79decffa5ef06b11c)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:25.244664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:25.308208 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87fbb253c94647b79decffa5ef06b11c', 'request_id': 'req-aa079fb0-3a4d-4f25-8215-d03ca68e6696'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:25.321646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87fbb253c94647b79decffa5ef06b11c', 'request_id': 'req-aa079fb0-3a4d-4f25-8215-d03ca68e6696'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:25.333965 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87fbb253c94647b79decffa5ef06b11c', 'request_id': 'req-aa079fb0-3a4d-4f25-8215-d03ca68e6696'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:25.346964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] The token cache is being invalidate because project 87fbb253c94647b79decffa5ef06b11c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:25.347701 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:25.348749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-aa079fb0-3a4d-4f25-8215-d03ca68e6696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:25.350796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1055/2112] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:25 2022] DELETE /identity/v3/projects/87fbb253c94647b79decffa5ef06b11c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:25.359254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dbe78-ee6c-480e-b175-094695bf5b17 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:25.359702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dbe78-ee6c-480e-b175-094695bf5b17 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:25.360140 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dbe78-ee6c-480e-b175-094695bf5b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:25.383608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-cd4dbe78-ee6c-480e-b175-094695bf5b17 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:25.384993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-cd4dbe78-ee6c-480e-b175-094695bf5b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:25.484068 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1058/2113] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:25.501458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bc881-e547-4f77-8c10-daceee9f5dad None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:25.501951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bc881-e547-4f77-8c10-daceee9f5dad None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:25.502379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bc881-e547-4f77-8c10-daceee9f5dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:25.539283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-5b4bc881-e547-4f77-8c10-daceee9f5dad None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:25.549739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-5b4bc881-e547-4f77-8c10-daceee9f5dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:25.567409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1056/2114] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:25.569135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dbe78-ee6c-480e-b175-094695bf5b17 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:25.582083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dbe78-ee6c-480e-b175-094695bf5b17 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:25.593340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d9912-0d00-49e5-81e6-3cc70e9efb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:25.594892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d9912-0d00-49e5-81e6-3cc70e9efb3e admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:25.595451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d9912-0d00-49e5-81e6-3cc70e9efb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:25.595989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d9912-0d00-49e5-81e6-3cc70e9efb3e admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:25.597469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d9912-0d00-49e5-81e6-3cc70e9efb3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:25.598042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d9912-0d00-49e5-81e6-3cc70e9efb3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:25.600970 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d9912-0d00-49e5-81e6-3cc70e9efb3e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:25.624919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1059/2115] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:56:25.626049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bc881-e547-4f77-8c10-daceee9f5dad None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:25.635902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bc881-e547-4f77-8c10-daceee9f5dad None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:25.641479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4b6b6-32a0-4e3f-aa23-638710b6cf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:25.642152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4b6b6-32a0-4e3f-aa23-638710b6cf33 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:25.642443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4b6b6-32a0-4e3f-aa23-638710b6cf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:25.642717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4b6b6-32a0-4e3f-aa23-638710b6cf33 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:25.643453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4b6b6-32a0-4e3f-aa23-638710b6cf33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:25.645523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4b6b6-32a0-4e3f-aa23-638710b6cf33 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:25.675181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1057/2116] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:25 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:25.679384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d9912-0d00-49e5-81e6-3cc70e9efb3e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:25.689648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d9912-0d00-49e5-81e6-3cc70e9efb3e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:25.755222 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6fb92-37a1-4866-8e8f-80266a9e43aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:25.755896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6fb92-37a1-4866-8e8f-80266a9e43aa admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:25.756180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6fb92-37a1-4866-8e8f-80266a9e43aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:25.756496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6fb92-37a1-4866-8e8f-80266a9e43aa admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:25.757190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6fb92-37a1-4866-8e8f-80266a9e43aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:25.759100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6fb92-37a1-4866-8e8f-80266a9e43aa admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:25.783094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1060/2117] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:25 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:56:25.787619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4b6b6-32a0-4e3f-aa23-638710b6cf33 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:25.798815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4b6b6-32a0-4e3f-aa23-638710b6cf33 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:25.803842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878acdb-63ff-4e19-9375-69b8dad00e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:25.804548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878acdb-63ff-4e19-9375-69b8dad00e4e admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:25.804833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878acdb-63ff-4e19-9375-69b8dad00e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:25.805103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878acdb-63ff-4e19-9375-69b8dad00e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:25.805828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878acdb-63ff-4e19-9375-69b8dad00e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:25.807437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878acdb-63ff-4e19-9375-69b8dad00e4e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:25.818941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1058/2118] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:25.824962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6fb92-37a1-4866-8e8f-80266a9e43aa admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:25.835720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6fb92-37a1-4866-8e8f-80266a9e43aa admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:25.840444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966902e0-3edf-486c-887e-f76ea227b55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:25.841110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966902e0-3edf-486c-887e-f76ea227b55e admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:25.841403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966902e0-3edf-486c-887e-f76ea227b55e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:25.841696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966902e0-3edf-486c-887e-f76ea227b55e admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:25.843003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966902e0-3edf-486c-887e-f76ea227b55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:25.844635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966902e0-3edf-486c-887e-f76ea227b55e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:25.855164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1061/2119] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:25.860268 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878acdb-63ff-4e19-9375-69b8dad00e4e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:25.870869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878acdb-63ff-4e19-9375-69b8dad00e4e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:25.877446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1ade6-b9e9-4469-b031-450ee7c4aec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:25.878382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1ade6-b9e9-4469-b031-450ee7c4aec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:25.878619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1ade6-b9e9-4469-b031-450ee7c4aec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:25.878899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1ade6-b9e9-4469-b031-450ee7c4aec9 admin admin] PATH_INFO: `/v3/projects/f390ec78a0d94d7fb5c1f5e94832be4d/users/530316303d8d4d089ad79c1e3e18bbee/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:25.894932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1ade6-b9e9-4469-b031-450ee7c4aec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f390ec78a0d94d7fb5c1f5e94832be4d, user_id=530316303d8d4d089ad79c1e3e18bbee, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:25.897629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1ade6-b9e9-4469-b031-450ee7c4aec9 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:25.912897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1059/2120] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:25 2022] PUT /identity/v3/projects/f390ec78a0d94d7fb5c1f5e94832be4d/users/530316303d8d4d089ad79c1e3e18bbee/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:25.925227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa978d-e16c-4cc8-84cf-d2e64a5abe24 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:25.925516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa978d-e16c-4cc8-84cf-d2e64a5abe24 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:25.925829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa978d-e16c-4cc8-84cf-d2e64a5abe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:25.955164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-21aa978d-e16c-4cc8-84cf-d2e64a5abe24 None None] MFA Rules not processed for user `530316303d8d4d089ad79c1e3e18bbee`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:25.956351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-21aa978d-e16c-4cc8-84cf-d2e64a5abe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:25.968821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:26.025949 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1062/2121] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:26.030580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1ade6-b9e9-4469-b031-450ee7c4aec9 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:26.060758 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:26.104954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1ade6-b9e9-4469-b031-450ee7c4aec9 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:26.109971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29fa91-704a-4a3d-a8d2-9dc08e3e99fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:26.110653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29fa91-704a-4a3d-a8d2-9dc08e3e99fe admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:26.110941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29fa91-704a-4a3d-a8d2-9dc08e3e99fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:26.111212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29fa91-704a-4a3d-a8d2-9dc08e3e99fe admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:26.111923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29fa91-704a-4a3d-a8d2-9dc08e3e99fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:26.113991 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29fa91-704a-4a3d-a8d2-9dc08e3e99fe admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:26.142159 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1060/2122] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:26 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:26.146616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa978d-e16c-4cc8-84cf-d2e64a5abe24 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:26.158588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa978d-e16c-4cc8-84cf-d2e64a5abe24 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:26.230069 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe3495-043c-4b75-94be-a91e73be3d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:26.230746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe3495-043c-4b75-94be-a91e73be3d40 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:26.231031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe3495-043c-4b75-94be-a91e73be3d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:26.231308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe3495-043c-4b75-94be-a91e73be3d40 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:26.231999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe3495-043c-4b75-94be-a91e73be3d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:26.234602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe3495-043c-4b75-94be-a91e73be3d40 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:26.255786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1063/2123] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:26 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:26.260249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29fa91-704a-4a3d-a8d2-9dc08e3e99fe admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:26.271946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29fa91-704a-4a3d-a8d2-9dc08e3e99fe admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:26.276972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3090957-3b74-4729-9184-949b26313b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:26.277637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3090957-3b74-4729-9184-949b26313b35 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:26.277917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3090957-3b74-4729-9184-949b26313b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:26.278183 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3090957-3b74-4729-9184-949b26313b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:26.278903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3090957-3b74-4729-9184-949b26313b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:26.281134 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3090957-3b74-4729-9184-949b26313b35 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:26.293045 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1061/2124] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:26.297345 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe3495-043c-4b75-94be-a91e73be3d40 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:26.308313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe3495-043c-4b75-94be-a91e73be3d40 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:26.313574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6c0b0-25ae-49d9-8949-5a2bcf59168c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:26.314724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6c0b0-25ae-49d9-8949-5a2bcf59168c admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:26.315129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6c0b0-25ae-49d9-8949-5a2bcf59168c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:26.315360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6c0b0-25ae-49d9-8949-5a2bcf59168c admin admin] PATH_INFO: `/v3/projects/7fc54060bb75444595002b2df06a4505/users/2d0569a3099a418dbad78e674839980a/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:26.332339 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6c0b0-25ae-49d9-8949-5a2bcf59168c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc54060bb75444595002b2df06a4505, user_id=2d0569a3099a418dbad78e674839980a, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:26.335027 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6c0b0-25ae-49d9-8949-5a2bcf59168c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:26.349141 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1064/2125] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:26 2022] PUT /identity/v3/projects/7fc54060bb75444595002b2df06a4505/users/2d0569a3099a418dbad78e674839980a/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:26.353112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3090957-3b74-4729-9184-949b26313b35 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:26.363200 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3090957-3b74-4729-9184-949b26313b35 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:26.427719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c431b-aed2-4a9e-93e2-b08a1ca17fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:26.428417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c431b-aed2-4a9e-93e2-b08a1ca17fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:26.428701 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c431b-aed2-4a9e-93e2-b08a1ca17fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:26.428974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c431b-aed2-4a9e-93e2-b08a1ca17fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:26.429688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c431b-aed2-4a9e-93e2-b08a1ca17fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:26.431297 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c431b-aed2-4a9e-93e2-b08a1ca17fc7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:26.441568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1062/2126] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:26.446068 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6c0b0-25ae-49d9-8949-5a2bcf59168c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:26.457407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6c0b0-25ae-49d9-8949-5a2bcf59168c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:26.462252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007c697-2af1-44e0-9ee4-1aa7f230c069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:26.463172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c697-2af1-44e0-9ee4-1aa7f230c069 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:26.463456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c697-2af1-44e0-9ee4-1aa7f230c069 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:26.463735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007c697-2af1-44e0-9ee4-1aa7f230c069 admin admin] PATH_INFO: `/v3/projects/7fc54060bb75444595002b2df06a4505/users/2d0569a3099a418dbad78e674839980a/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:26.465783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007c697-2af1-44e0-9ee4-1aa7f230c069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc54060bb75444595002b2df06a4505, user_id=2d0569a3099a418dbad78e674839980a, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:26.469181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007c697-2af1-44e0-9ee4-1aa7f230c069 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:26.479879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1065/2127] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:26 2022] PUT /identity/v3/projects/7fc54060bb75444595002b2df06a4505/users/2d0569a3099a418dbad78e674839980a/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:26.490970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ba766-f37d-4718-a42a-a6b01459fc60 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:26.491312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ba766-f37d-4718-a42a-a6b01459fc60 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:26.491540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ba766-f37d-4718-a42a-a6b01459fc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:26.518825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4e1ba766-f37d-4718-a42a-a6b01459fc60 None None] MFA Rules not processed for user `2d0569a3099a418dbad78e674839980a`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:26.520058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4e1ba766-f37d-4718-a42a-a6b01459fc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:26.590669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1063/2128] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:26.603440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007c697-2af1-44e0-9ee4-1aa7f230c069 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:26.605303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f007c697-2af1-44e0-9ee4-1aa7f230c069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:26.665337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007c697-2af1-44e0-9ee4-1aa7f230c069 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:26.669280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734b85f-6887-4a9a-9346-4d51741db07c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:26.669933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734b85f-6887-4a9a-9346-4d51741db07c service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:26.670221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734b85f-6887-4a9a-9346-4d51741db07c service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:26.670538 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734b85f-6887-4a9a-9346-4d51741db07c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:26.681159 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734b85f-6887-4a9a-9346-4d51741db07c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:26.682601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734b85f-6887-4a9a-9346-4d51741db07c service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:26.695148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1066/2129] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:56:26.777984 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ba766-f37d-4718-a42a-a6b01459fc60 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:26.787633 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ba766-f37d-4718-a42a-a6b01459fc60 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:26.791286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8d687-2787-45ea-b853-a46006b00320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:26.791931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8d687-2787-45ea-b853-a46006b00320 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:26.792305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8d687-2787-45ea-b853-a46006b00320 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:26.792520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8d687-2787-45ea-b853-a46006b00320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:26.817094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:26.859977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8d687-2787-45ea-b853-a46006b00320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:26.861652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8d687-2787-45ea-b853-a46006b00320 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:26.871319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1064/2130] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:36.690803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:36.866995 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=19 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:37.538982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734b85f-6887-4a9a-9346-4d51741db07c service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:37.540482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7734b85f-6887-4a9a-9346-4d51741db07c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:37.627350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734b85f-6887-4a9a-9346-4d51741db07c service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:37.632846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:37.633843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:37.634062 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:37.634349 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] PATH_INFO: `/v3/users/7242a599a607400eae2dd49c7cbf07e4` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:37.635974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7242a599a607400eae2dd49c7cbf07e4)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:37.638303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:37.692701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7242a599a607400eae2dd49c7cbf07e4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:37.707310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7242a599a607400eae2dd49c7cbf07e4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:37.708711 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7242a599a607400eae2dd49c7cbf07e4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:37.729167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7242a599a607400eae2dd49c7cbf07e4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:37.738886 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7242a599a607400eae2dd49c7cbf07e4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:37.746032 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1067/2131] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:37 2022] DELETE /identity/v3/users/7242a599a607400eae2dd49c7cbf07e4 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:37.752516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e248c2c-24ec-494e-995d-2fa416cbc135 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:37.753011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e248c2c-24ec-494e-995d-2fa416cbc135 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:37.753535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e248c2c-24ec-494e-995d-2fa416cbc135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:37.776008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3e248c2c-24ec-494e-995d-2fa416cbc135 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:37.777533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3e248c2c-24ec-494e-995d-2fa416cbc135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:37.851359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1065/2132] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:37.867926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:37.872303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:37.994966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79266-a872-4a70-999a-3afc49541bb9 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:38.001200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b85f75-41ad-47a8-b5ed-47cb7a4aa147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:38.002925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b85f75-41ad-47a8-b5ed-47cb7a4aa147 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:38.003506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b85f75-41ad-47a8-b5ed-47cb7a4aa147 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:38.003867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b85f75-41ad-47a8-b5ed-47cb7a4aa147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:38.022254 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b85f75-41ad-47a8-b5ed-47cb7a4aa147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:38.023608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b85f75-41ad-47a8-b5ed-47cb7a4aa147 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:38.039765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1068/2133] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:38.199281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e248c2c-24ec-494e-995d-2fa416cbc135 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:38.201402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3e248c2c-24ec-494e-995d-2fa416cbc135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:38.216467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e248c2c-24ec-494e-995d-2fa416cbc135 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:38.225908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:38.227192 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:38.227546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:38.227859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] PATH_INFO: `/v3/users/530316303d8d4d089ad79c1e3e18bbee` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:38.234176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=530316303d8d4d089ad79c1e3e18bbee)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:38.237525 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:38.353917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '530316303d8d4d089ad79c1e3e18bbee'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:38.370415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '530316303d8d4d089ad79c1e3e18bbee'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:38.380073 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '530316303d8d4d089ad79c1e3e18bbee'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:38.380848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '530316303d8d4d089ad79c1e3e18bbee'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:38.397400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '530316303d8d4d089ad79c1e3e18bbee'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:38.406825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1066/2134] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:38 2022] DELETE /identity/v3/users/530316303d8d4d089ad79c1e3e18bbee => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:38.411853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2969d1-b9f5-402d-83d1-51270e986080 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:38.412154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2969d1-b9f5-402d-83d1-51270e986080 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:38.412472 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2969d1-b9f5-402d-83d1-51270e986080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:38.434539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-4a2969d1-b9f5-402d-83d1-51270e986080 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:38.435962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4a2969d1-b9f5-402d-83d1-51270e986080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:38.458081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:38.468745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:38.470559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:38.543362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1069/2135] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:38.554679 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2969d1-b9f5-402d-83d1-51270e986080 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:38.556471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4a2969d1-b9f5-402d-83d1-51270e986080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:38.574282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e10ce5-0805-4632-9569-fa189fd360dd admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:38.586259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:38.589135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:38.589497 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:38.589864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] PATH_INFO: `/v3/projects/0402894d9a08472aae07d5c07ff0c966` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:38.591468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0402894d9a08472aae07d5c07ff0c966)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:38.594135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:38.621668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:38.645603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2969d1-b9f5-402d-83d1-51270e986080 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:38.651413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a67e5-9f50-46a9-8222-a7f772266e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:38.652643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a67e5-9f50-46a9-8222-a7f772266e97 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:38.653298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a67e5-9f50-46a9-8222-a7f772266e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:38.653705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a67e5-9f50-46a9-8222-a7f772266e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:38.667165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a67e5-9f50-46a9-8222-a7f772266e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:38.669249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a67e5-9f50-46a9-8222-a7f772266e97 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:38.682617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1070/2136] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:56:38.698685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0402894d9a08472aae07d5c07ff0c966', 'request_id': 'req-4034d4c6-6ce1-4698-a50d-979fc865cb38'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:38.733480 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0402894d9a08472aae07d5c07ff0c966', 'request_id': 'req-4034d4c6-6ce1-4698-a50d-979fc865cb38'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:38.742569 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0402894d9a08472aae07d5c07ff0c966', 'request_id': 'req-4034d4c6-6ce1-4698-a50d-979fc865cb38'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:38.756539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] The token cache is being invalidate because project 0402894d9a08472aae07d5c07ff0c966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:38.756815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:38.757541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:38.760034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1067/2137] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:38 2022] DELETE /identity/v3/projects/0402894d9a08472aae07d5c07ff0c966 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:38.765737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a67e5-9f50-46a9-8222-a7f772266e97 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:38.791941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a67e5-9f50-46a9-8222-a7f772266e97 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:38.921749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:38.928645 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:38.928645 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:38.928645 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] PATH_INFO: `/v3/users/36eb245700294f9889926d68adfa483e` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:38.930616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36eb245700294f9889926d68adfa483e)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:38.939099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:39.056314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36eb245700294f9889926d68adfa483e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.070185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36eb245700294f9889926d68adfa483e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.071635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36eb245700294f9889926d68adfa483e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.100748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:39.122451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36eb245700294f9889926d68adfa483e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.140528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36eb245700294f9889926d68adfa483e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.149226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1071/2138] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:38 2022] DELETE /identity/v3/users/36eb245700294f9889926d68adfa483e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:39.253645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:39.256267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:39.274811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034d4c6-6ce1-4698-a50d-979fc865cb38 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:39.409516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:39.411049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:39.411648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:39.412121 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] PATH_INFO: `/v3/projects/f390ec78a0d94d7fb5c1f5e94832be4d` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:39.413979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f390ec78a0d94d7fb5c1f5e94832be4d)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:39.416594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:39.502959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f390ec78a0d94d7fb5c1f5e94832be4d', 'request_id': 'req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.514307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f390ec78a0d94d7fb5c1f5e94832be4d', 'request_id': 'req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.521817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f390ec78a0d94d7fb5c1f5e94832be4d', 'request_id': 'req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.537131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] The token cache is being invalidate because project f390ec78a0d94d7fb5c1f5e94832be4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:39.537676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.538535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.540136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1068/2139] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:39 2022] DELETE /identity/v3/projects/f390ec78a0d94d7fb5c1f5e94832be4d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:39.544301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:39.546530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:39.563060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474ebbe-4455-4b42-b3b0-fcc83e7c67f7 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:39.675032 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3e834-29ad-485e-bd0b-14a3bc57d468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:39.676064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e834-29ad-485e-bd0b-14a3bc57d468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:39.676418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e834-29ad-485e-bd0b-14a3bc57d468 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:39.676704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e834-29ad-485e-bd0b-14a3bc57d468 admin admin] PATH_INFO: `/v3/users/2d0569a3099a418dbad78e674839980a` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:39.678046 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3e834-29ad-485e-bd0b-14a3bc57d468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d0569a3099a418dbad78e674839980a)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:39.680435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3e834-29ad-485e-bd0b-14a3bc57d468 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:39.755924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-41d3e834-29ad-485e-bd0b-14a3bc57d468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d0569a3099a418dbad78e674839980a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.765583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-41d3e834-29ad-485e-bd0b-14a3bc57d468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d0569a3099a418dbad78e674839980a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.766317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-41d3e834-29ad-485e-bd0b-14a3bc57d468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d0569a3099a418dbad78e674839980a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.788146 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-41d3e834-29ad-485e-bd0b-14a3bc57d468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d0569a3099a418dbad78e674839980a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.806099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-41d3e834-29ad-485e-bd0b-14a3bc57d468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d0569a3099a418dbad78e674839980a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:39.820604 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1072/2140] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:39 2022] DELETE /identity/v3/users/2d0569a3099a418dbad78e674839980a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:39.896230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:39.898434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:39.931172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9d3dc-f798-4661-8b70-5f0d4e6e07e3 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:39.995719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:40.035443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:40.036400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:40.036694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:40.036971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] PATH_INFO: `/v3/projects/d8abe092b88a4ca8adb8d7ab84bf6468` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:40.038362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8abe092b88a4ca8adb8d7ab84bf6468)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:40.040619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:40.104690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8abe092b88a4ca8adb8d7ab84bf6468', 'request_id': 'req-c3f54012-afa2-408d-b785-c6214f4d050a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:40.115008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8abe092b88a4ca8adb8d7ab84bf6468', 'request_id': 'req-c3f54012-afa2-408d-b785-c6214f4d050a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:40.131485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8abe092b88a4ca8adb8d7ab84bf6468', 'request_id': 'req-c3f54012-afa2-408d-b785-c6214f4d050a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:40.146392 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] The token cache is being invalidate because project d8abe092b88a4ca8adb8d7ab84bf6468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:40.147080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:40.148226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:40.151418 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1069/2141] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:39 2022] DELETE /identity/v3/projects/d8abe092b88a4ca8adb8d7ab84bf6468 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:40.159978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c8c8-9ff8-437c-8073-8a4f3b109a6d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:40.160478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c8c8-9ff8-437c-8073-8a4f3b109a6d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:40.160965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2c8c8-9ff8-437c-8073-8a4f3b109a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:40.174130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:40.175632 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:40.196244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-34d2c8c8-9ff8-437c-8073-8a4f3b109a6d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:40.198587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-34d2c8c8-9ff8-437c-8073-8a4f3b109a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:40.297784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f54012-afa2-408d-b785-c6214f4d050a admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:40.304085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da92cae-fcc1-4e5b-8777-f8e3fdc728b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:40.305120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da92cae-fcc1-4e5b-8777-f8e3fdc728b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:40.305446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da92cae-fcc1-4e5b-8777-f8e3fdc728b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:40.305998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da92cae-fcc1-4e5b-8777-f8e3fdc728b5 admin admin] PATH_INFO: `/v3/users/6688fdba27694ea0bc3e208e30cd2a26` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:40.308407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da92cae-fcc1-4e5b-8777-f8e3fdc728b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6688fdba27694ea0bc3e208e30cd2a26)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:40.309749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da92cae-fcc1-4e5b-8777-f8e3fdc728b5 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:40.357021 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1073/2142] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:40.368009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06681cc-c050-4236-85f9-ae884283cd02 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:40.368447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06681cc-c050-4236-85f9-ae884283cd02 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:40.368929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06681cc-c050-4236-85f9-ae884283cd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:40.382777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:40.386490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6da92cae-fcc1-4e5b-8777-f8e3fdc728b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6688fdba27694ea0bc3e208e30cd2a26'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:40.400487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6da92cae-fcc1-4e5b-8777-f8e3fdc728b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6688fdba27694ea0bc3e208e30cd2a26'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:40.406260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c06681cc-c050-4236-85f9-ae884283cd02 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:40.409386 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c06681cc-c050-4236-85f9-ae884283cd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:40.412610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6da92cae-fcc1-4e5b-8777-f8e3fdc728b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6688fdba27694ea0bc3e208e30cd2a26'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:40.413450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6da92cae-fcc1-4e5b-8777-f8e3fdc728b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6688fdba27694ea0bc3e208e30cd2a26'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:40.416988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:40.423793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1074/2143] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:40.429243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06681cc-c050-4236-85f9-ae884283cd02 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:40.445493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06681cc-c050-4236-85f9-ae884283cd02 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:40.454728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201fcf0-3447-4ba8-a53f-2e6fe12dda0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:40.455458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201fcf0-3447-4ba8-a53f-2e6fe12dda0f admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:40.455812 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6da92cae-fcc1-4e5b-8777-f8e3fdc728b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6688fdba27694ea0bc3e208e30cd2a26'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:40.455926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201fcf0-3447-4ba8-a53f-2e6fe12dda0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:40.456234 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201fcf0-3447-4ba8-a53f-2e6fe12dda0f admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:40.457389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201fcf0-3447-4ba8-a53f-2e6fe12dda0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:40.457712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201fcf0-3447-4ba8-a53f-2e6fe12dda0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:40.459736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201fcf0-3447-4ba8-a53f-2e6fe12dda0f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:40.474190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1070/2144] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:40 2022] DELETE /identity/v3/users/6688fdba27694ea0bc3e208e30cd2a26 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:40.477878 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2fc76-c28d-4328-a02c-0ebbb42d57dc None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:40.478213 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2fc76-c28d-4328-a02c-0ebbb42d57dc None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:40.478542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2fc76-c28d-4328-a02c-0ebbb42d57dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:40.483925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1075/2145] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:56:40.496365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201fcf0-3447-4ba8-a53f-2e6fe12dda0f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:40.508447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201fcf0-3447-4ba8-a53f-2e6fe12dda0f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:40.519760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-5af2fc76-c28d-4328-a02c-0ebbb42d57dc None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:40.521626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-5af2fc76-c28d-4328-a02c-0ebbb42d57dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:40.601464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a04f4c-0b62-4b84-b9a8-7fbbc0ede7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:40.603534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a04f4c-0b62-4b84-b9a8-7fbbc0ede7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:40.603821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a04f4c-0b62-4b84-b9a8-7fbbc0ede7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:40.604129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a04f4c-0b62-4b84-b9a8-7fbbc0ede7f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:40.604887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a04f4c-0b62-4b84-b9a8-7fbbc0ede7f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:40.614632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a04f4c-0b62-4b84-b9a8-7fbbc0ede7f7 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:40.642277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1071/2146] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:40.657482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2fc76-c28d-4328-a02c-0ebbb42d57dc None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:40.659837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-5af2fc76-c28d-4328-a02c-0ebbb42d57dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:40.664043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1076/2147] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:40 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:40.669072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a04f4c-0b62-4b84-b9a8-7fbbc0ede7f7 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:40.746607 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2fc76-c28d-4328-a02c-0ebbb42d57dc None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:40.810039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a04f4c-0b62-4b84-b9a8-7fbbc0ede7f7 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:40.822857 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584af16-ec16-4a66-b309-f08596c1d2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:40.823850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584af16-ec16-4a66-b309-f08596c1d2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:40.824262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584af16-ec16-4a66-b309-f08596c1d2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:40.824718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584af16-ec16-4a66-b309-f08596c1d2d9 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:40.825619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584af16-ec16-4a66-b309-f08596c1d2d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:40.828152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584af16-ec16-4a66-b309-f08596c1d2d9 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:40.877282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1077/2148] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:40 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:40.877282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584af16-ec16-4a66-b309-f08596c1d2d9 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:40.880378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7584af16-ec16-4a66-b309-f08596c1d2d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:40.894685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea595c69-15ad-4117-9c74-e8ae07269c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:40.897036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595c69-15ad-4117-9c74-e8ae07269c17 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:40.897485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595c69-15ad-4117-9c74-e8ae07269c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:40.897934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595c69-15ad-4117-9c74-e8ae07269c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:40.901039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584af16-ec16-4a66-b309-f08596c1d2d9 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:40.914107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:40.915502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:40.915884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:40.916307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] PATH_INFO: `/v3/projects/7fc54060bb75444595002b2df06a4505` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:40.917994 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc54060bb75444595002b2df06a4505)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:40.918476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea595c69-15ad-4117-9c74-e8ae07269c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:40.921050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea595c69-15ad-4117-9c74-e8ae07269c17 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:40.921496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:40.938788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1072/2149] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:40.940202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea595c69-15ad-4117-9c74-e8ae07269c17 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:40.953848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea595c69-15ad-4117-9c74-e8ae07269c17 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:40.963421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d74c6-ff64-4c1f-93b3-1e7ee3c70518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:40.964399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d74c6-ff64-4c1f-93b3-1e7ee3c70518 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:40.964827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d74c6-ff64-4c1f-93b3-1e7ee3c70518 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:40.965244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d74c6-ff64-4c1f-93b3-1e7ee3c70518 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:40.967314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d74c6-ff64-4c1f-93b3-1e7ee3c70518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:40.969129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d74c6-ff64-4c1f-93b3-1e7ee3c70518 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:40.987245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1073/2150] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:40.992560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d74c6-ff64-4c1f-93b3-1e7ee3c70518 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.016356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d74c6-ff64-4c1f-93b3-1e7ee3c70518 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:41.032917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fcf1f-1c4c-4193-be9d-6b6426579a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:41.033936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fcf1f-1c4c-4193-be9d-6b6426579a15 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.034410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fcf1f-1c4c-4193-be9d-6b6426579a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.034867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fcf1f-1c4c-4193-be9d-6b6426579a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.035897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fcf1f-1c4c-4193-be9d-6b6426579a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:41.038534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fcf1f-1c4c-4193-be9d-6b6426579a15 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:41.045071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc54060bb75444595002b2df06a4505', 'request_id': 'req-c685566f-cc67-4942-8b42-b887a8f7b501'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:41.057223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:41.057325 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1074/2151] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:41.065117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fcf1f-1c4c-4193-be9d-6b6426579a15 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.065902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc54060bb75444595002b2df06a4505', 'request_id': 'req-c685566f-cc67-4942-8b42-b887a8f7b501'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:41.079064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc54060bb75444595002b2df06a4505', 'request_id': 'req-c685566f-cc67-4942-8b42-b887a8f7b501'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:41.090404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fcf1f-1c4c-4193-be9d-6b6426579a15 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:41.101249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] The token cache is being invalidate because project 7fc54060bb75444595002b2df06a4505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:41.101902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:41.103031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-c685566f-cc67-4942-8b42-b887a8f7b501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:41.105081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1078/2152] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:40 2022] DELETE /identity/v3/projects/7fc54060bb75444595002b2df06a4505 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:41.112657 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e491f3-311a-4c8b-8282-b292c2d75cee None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.113033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e491f3-311a-4c8b-8282-b292c2d75cee None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.113445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e491f3-311a-4c8b-8282-b292c2d75cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.136940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-89e491f3-311a-4c8b-8282-b292c2d75cee None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:41.138385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-89e491f3-311a-4c8b-8282-b292c2d75cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:41.143176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:41.217188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f86ac-4589-43f6-a5fe-2bc06b8d6b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:41.218196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f86ac-4589-43f6-a5fe-2bc06b8d6b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.218559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f86ac-4589-43f6-a5fe-2bc06b8d6b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.218912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f86ac-4589-43f6-a5fe-2bc06b8d6b7c admin admin] PATH_INFO: `/v3/projects/b5c3218e26bd4f30baf4943275eb35a1/users/68612f130bb1478bb75da751791a2411/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.239022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f86ac-4589-43f6-a5fe-2bc06b8d6b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5c3218e26bd4f30baf4943275eb35a1, user_id=68612f130bb1478bb75da751791a2411, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:41.245132 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f86ac-4589-43f6-a5fe-2bc06b8d6b7c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:41.247119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1079/2153] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:41.257939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7c9bb-45b7-457e-93ce-307162840924 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.258628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7c9bb-45b7-457e-93ce-307162840924 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.259223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7c9bb-45b7-457e-93ce-307162840924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.265309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1075/2154] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:41 2022] PUT /identity/v3/projects/b5c3218e26bd4f30baf4943275eb35a1/users/68612f130bb1478bb75da751791a2411/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:41.277279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479be95-862d-432d-8f76-af776f2d8978 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.277948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479be95-862d-432d-8f76-af776f2d8978 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.278335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479be95-862d-432d-8f76-af776f2d8978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.285452 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-aea7c9bb-45b7-457e-93ce-307162840924 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:41.287535 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-aea7c9bb-45b7-457e-93ce-307162840924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:41.324201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-2479be95-862d-432d-8f76-af776f2d8978 None None] MFA Rules not processed for user `68612f130bb1478bb75da751791a2411`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:41.327416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2479be95-862d-432d-8f76-af776f2d8978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:41.390215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1080/2155] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:41.394933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7c9bb-45b7-457e-93ce-307162840924 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.408790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7c9bb-45b7-457e-93ce-307162840924 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:41.416212 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fce87-2325-454e-a85b-6671830d37cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:41.417567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fce87-2325-454e-a85b-6671830d37cf admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.418751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fce87-2325-454e-a85b-6671830d37cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.419035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fce87-2325-454e-a85b-6671830d37cf admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.419798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fce87-2325-454e-a85b-6671830d37cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:41.421703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fce87-2325-454e-a85b-6671830d37cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:41.424018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fce87-2325-454e-a85b-6671830d37cf admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:41.444128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1081/2156] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:56:41.445516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fce87-2325-454e-a85b-6671830d37cf admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.456495 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1076/2157] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:41.461164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479be95-862d-432d-8f76-af776f2d8978 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.462694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2479be95-862d-432d-8f76-af776f2d8978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:41.464567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fce87-2325-454e-a85b-6671830d37cf admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:41.474953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c16299-02e9-48fa-bbf1-4432ea82ee82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:41.474953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16299-02e9-48fa-bbf1-4432ea82ee82 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.474953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16299-02e9-48fa-bbf1-4432ea82ee82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.474953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c16299-02e9-48fa-bbf1-4432ea82ee82 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.475759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c16299-02e9-48fa-bbf1-4432ea82ee82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:41.477853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c16299-02e9-48fa-bbf1-4432ea82ee82 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:41.479289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479be95-862d-432d-8f76-af776f2d8978 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:41.485765 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d94ff-2b44-4365-ad54-5cdace8be6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:41.486537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d94ff-2b44-4365-ad54-5cdace8be6ad admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.486828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d94ff-2b44-4365-ad54-5cdace8be6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.487597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d94ff-2b44-4365-ad54-5cdace8be6ad admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.488056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d94ff-2b44-4365-ad54-5cdace8be6ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:41.490411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d94ff-2b44-4365-ad54-5cdace8be6ad admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:41.523455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1082/2158] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:41 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:41.527695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c16299-02e9-48fa-bbf1-4432ea82ee82 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.539445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1077/2159] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:41 2022] POST /identity/v3/projects => generated 387 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:56:41.542243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d94ff-2b44-4365-ad54-5cdace8be6ad admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.590310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:41.611320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c16299-02e9-48fa-bbf1-4432ea82ee82 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:41.643688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d94ff-2b44-4365-ad54-5cdace8be6ad admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:41.650439 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a39e71-0376-4eb6-b845-ac9300322d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:41.651022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a39e71-0376-4eb6-b845-ac9300322d1c admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.651677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a39e71-0376-4eb6-b845-ac9300322d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.651790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a39e71-0376-4eb6-b845-ac9300322d1c admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.653104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a39e71-0376-4eb6-b845-ac9300322d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:41.657557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a39e71-0376-4eb6-b845-ac9300322d1c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:41.697301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:41.711627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1078/2160] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:41 2022] POST /identity/v3/users => generated 300 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:41.737632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51b258-5ee0-4d47-b1ea-b9994bb403e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:41.738330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51b258-5ee0-4d47-b1ea-b9994bb403e7 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.738614 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51b258-5ee0-4d47-b1ea-b9994bb403e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.738900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51b258-5ee0-4d47-b1ea-b9994bb403e7 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.739621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51b258-5ee0-4d47-b1ea-b9994bb403e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:41.741895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51b258-5ee0-4d47-b1ea-b9994bb403e7 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:41.743590 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a39e71-0376-4eb6-b845-ac9300322d1c admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.750045 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f8a39e71-0376-4eb6-b845-ac9300322d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:41.780969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a39e71-0376-4eb6-b845-ac9300322d1c admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:41.788826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:41.789540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1083/2161] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:41 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:56:41.789680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.789866 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.790195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] PATH_INFO: `/v3/projects/a05f2ab99a1b4a6b9190ac7b513f7896` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.790402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51b258-5ee0-4d47-b1ea-b9994bb403e7 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.791430 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a05f2ab99a1b4a6b9190ac7b513f7896)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:41.793450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:41.801268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51b258-5ee0-4d47-b1ea-b9994bb403e7 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:41.812369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159fbf6-81fd-4d5c-b501-966f9feab7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:41.813099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159fbf6-81fd-4d5c-b501-966f9feab7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.813356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159fbf6-81fd-4d5c-b501-966f9feab7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.813640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159fbf6-81fd-4d5c-b501-966f9feab7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.814660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159fbf6-81fd-4d5c-b501-966f9feab7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:41.816294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1159fbf6-81fd-4d5c-b501-966f9feab7a0 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:41.832173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1084/2162] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:41.833540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159fbf6-81fd-4d5c-b501-966f9feab7a0 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.846689 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159fbf6-81fd-4d5c-b501-966f9feab7a0 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:41.852461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bceb9-6746-43df-82e5-0620493da962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:41.853380 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bceb9-6746-43df-82e5-0620493da962 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.854169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bceb9-6746-43df-82e5-0620493da962 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.854169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bceb9-6746-43df-82e5-0620493da962 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:41.854698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0bceb9-6746-43df-82e5-0620493da962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:41.856374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0bceb9-6746-43df-82e5-0620493da962 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:41.871944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1085/2163] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:41.875985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bceb9-6746-43df-82e5-0620493da962 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.881700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a05f2ab99a1b4a6b9190ac7b513f7896', 'request_id': 'req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:41.892549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bceb9-6746-43df-82e5-0620493da962 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:41.899971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a05f2ab99a1b4a6b9190ac7b513f7896', 'request_id': 'req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:41.909392 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a05f2ab99a1b4a6b9190ac7b513f7896', 'request_id': 'req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:41.924234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] The token cache is being invalidate because project a05f2ab99a1b4a6b9190ac7b513f7896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:41.924745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:41.925636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:41.932196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1079/2164] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:41 2022] DELETE /identity/v3/projects/a05f2ab99a1b4a6b9190ac7b513f7896 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:41.933637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:41.937090 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:41.980185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425846d3-c819-400f-8631-f138b1821bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:41.981159 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425846d3-c819-400f-8631-f138b1821bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:41.981516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425846d3-c819-400f-8631-f138b1821bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:41.981874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425846d3-c819-400f-8631-f138b1821bb7 admin admin] PATH_INFO: `/v3/projects/d03dc47dd1fc4766a123e56efcbdffaf/users/bd70a233f1eb4ca08bd404919d55c9e1/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.009888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425846d3-c819-400f-8631-f138b1821bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03dc47dd1fc4766a123e56efcbdffaf, user_id=bd70a233f1eb4ca08bd404919d55c9e1, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.012537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc24a2-01b7-4bed-b7fe-de77dcf2c269 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.013435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425846d3-c819-400f-8631-f138b1821bb7 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.019007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2817d-daee-407f-89f0-860ffc239ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.019893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2817d-daee-407f-89f0-860ffc239ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.021421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2817d-daee-407f-89f0-860ffc239ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.021847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2817d-daee-407f-89f0-860ffc239ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.022813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2817d-daee-407f-89f0-860ffc239ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.024702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2817d-daee-407f-89f0-860ffc239ad7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.035861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1086/2165] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:41 2022] PUT /identity/v3/projects/d03dc47dd1fc4766a123e56efcbdffaf/users/bd70a233f1eb4ca08bd404919d55c9e1/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:42.037933 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1080/2166] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:42.038151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86ae87-4636-439b-adb4-8ddc8b9be01c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.038459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86ae87-4636-439b-adb4-8ddc8b9be01c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.038767 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86ae87-4636-439b-adb4-8ddc8b9be01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.040150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2817d-daee-407f-89f0-860ffc239ad7 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.042315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-33c2817d-daee-407f-89f0-860ffc239ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:42.058321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2817d-daee-407f-89f0-860ffc239ad7 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.065266 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-fc86ae87-4636-439b-adb4-8ddc8b9be01c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:42.067059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fc86ae87-4636-439b-adb4-8ddc8b9be01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:42.157632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:42.164967 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf491253-c1d6-471f-a298-d6cafd41c284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.166535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf491253-c1d6-471f-a298-d6cafd41c284 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.166900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf491253-c1d6-471f-a298-d6cafd41c284 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.168570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf491253-c1d6-471f-a298-d6cafd41c284 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.168570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf491253-c1d6-471f-a298-d6cafd41c284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.171715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf491253-c1d6-471f-a298-d6cafd41c284 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.189603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1081/2167] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:42.191483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf491253-c1d6-471f-a298-d6cafd41c284 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.191850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1087/2168] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:42.192752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86ae87-4636-439b-adb4-8ddc8b9be01c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.210156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86ae87-4636-439b-adb4-8ddc8b9be01c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.220900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e93c2d-cfc3-4243-85d5-7288e12c36fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.222644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e93c2d-cfc3-4243-85d5-7288e12c36fc admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.223019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e93c2d-cfc3-4243-85d5-7288e12c36fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.223895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e93c2d-cfc3-4243-85d5-7288e12c36fc admin admin] PATH_INFO: `/v3/projects/d03dc47dd1fc4766a123e56efcbdffaf/users/bd70a233f1eb4ca08bd404919d55c9e1/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.224774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf491253-c1d6-471f-a298-d6cafd41c284 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.227132 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e93c2d-cfc3-4243-85d5-7288e12c36fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03dc47dd1fc4766a123e56efcbdffaf, user_id=bd70a233f1eb4ca08bd404919d55c9e1, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.230316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e93c2d-cfc3-4243-85d5-7288e12c36fc admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.233724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255a772-bc2d-4a9f-88e2-a5a4fefd9d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.234708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255a772-bc2d-4a9f-88e2-a5a4fefd9d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.235004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255a772-bc2d-4a9f-88e2-a5a4fefd9d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.235338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255a772-bc2d-4a9f-88e2-a5a4fefd9d4c admin admin] PATH_INFO: `/v3/projects/b3e57460f4dd42ce812e3f304d1932ed/users/34a5a1913029485c9b4b38a8b4b8e149/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.251919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1088/2169] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:42 2022] PUT /identity/v3/projects/d03dc47dd1fc4766a123e56efcbdffaf/users/bd70a233f1eb4ca08bd404919d55c9e1/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:42.254990 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7269-2db0-425b-9c1f-ad3676f8cdc4 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.255147 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:42.255708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7269-2db0-425b-9c1f-ad3676f8cdc4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.256092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7269-2db0-425b-9c1f-ad3676f8cdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.263693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255a772-bc2d-4a9f-88e2-a5a4fefd9d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3e57460f4dd42ce812e3f304d1932ed, user_id=34a5a1913029485c9b4b38a8b4b8e149, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.267625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255a772-bc2d-4a9f-88e2-a5a4fefd9d4c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.289013 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ecdf7269-2db0-425b-9c1f-ad3676f8cdc4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:42.290805 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ecdf7269-2db0-425b-9c1f-ad3676f8cdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:42.290932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1082/2170] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:42 2022] PUT /identity/v3/projects/b3e57460f4dd42ce812e3f304d1932ed/users/34a5a1913029485c9b4b38a8b4b8e149/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:42.292458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236280d-f5de-4708-8997-8799297ef1bb None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.292942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236280d-f5de-4708-8997-8799297ef1bb None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.293077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236280d-f5de-4708-8997-8799297ef1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.341775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-8236280d-f5de-4708-8997-8799297ef1bb None None] MFA Rules not processed for user `bd70a233f1eb4ca08bd404919d55c9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:42.344559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8236280d-f5de-4708-8997-8799297ef1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:42.387405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1089/2171] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:42.390540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede9584-9968-445e-b39a-df928bb575f4 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.390540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede9584-9968-445e-b39a-df928bb575f4 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.392650 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede9584-9968-445e-b39a-df928bb575f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.429328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-6ede9584-9968-445e-b39a-df928bb575f4 None None] MFA Rules not processed for user `34a5a1913029485c9b4b38a8b4b8e149`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:42.430589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6ede9584-9968-445e-b39a-df928bb575f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:42.452769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1083/2172] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:42.453970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236280d-f5de-4708-8997-8799297ef1bb None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.480694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236280d-f5de-4708-8997-8799297ef1bb None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.486428 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad2d2b-037c-49fe-943d-bc7ad0650e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.487762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad2d2b-037c-49fe-943d-bc7ad0650e46 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.488676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad2d2b-037c-49fe-943d-bc7ad0650e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.489526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad2d2b-037c-49fe-943d-bc7ad0650e46 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.490180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad2d2b-037c-49fe-943d-bc7ad0650e46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:42.490547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad2d2b-037c-49fe-943d-bc7ad0650e46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.493015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad2d2b-037c-49fe-943d-bc7ad0650e46 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.517928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1084/2173] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:56:42.519037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad2d2b-037c-49fe-943d-bc7ad0650e46 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.521753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dfad2d2b-037c-49fe-943d-bc7ad0650e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:42.547432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1090/2174] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:42.548438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede9584-9968-445e-b39a-df928bb575f4 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.566450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede9584-9968-445e-b39a-df928bb575f4 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.578351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd31fd-ffb6-41a6-941c-0968e512fe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.579062 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd31fd-ffb6-41a6-941c-0968e512fe68 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.579371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd31fd-ffb6-41a6-941c-0968e512fe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.579740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd31fd-ffb6-41a6-941c-0968e512fe68 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.580568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd31fd-ffb6-41a6-941c-0968e512fe68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.582721 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd31fd-ffb6-41a6-941c-0968e512fe68 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.611164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad2d2b-037c-49fe-943d-bc7ad0650e46 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.616494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796e5d7-8a88-4804-8f30-fffa25b7aca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.617474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796e5d7-8a88-4804-8f30-fffa25b7aca0 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.617993 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796e5d7-8a88-4804-8f30-fffa25b7aca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.618324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796e5d7-8a88-4804-8f30-fffa25b7aca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.624575 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1091/2175] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:42 2022] POST /identity/v3/projects => generated 369 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:42.625679 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd31fd-ffb6-41a6-941c-0968e512fe68 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.632088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796e5d7-8a88-4804-8f30-fffa25b7aca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.634127 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796e5d7-8a88-4804-8f30-fffa25b7aca0 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.647666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1085/2176] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:42.648898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796e5d7-8a88-4804-8f30-fffa25b7aca0 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.665014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796e5d7-8a88-4804-8f30-fffa25b7aca0 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.696005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:42.727074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd31fd-ffb6-41a6-941c-0968e512fe68 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.733765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df10337-39b9-4abc-8be2-ece9d4213d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.734851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df10337-39b9-4abc-8be2-ece9d4213d0b admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.735435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df10337-39b9-4abc-8be2-ece9d4213d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.735846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df10337-39b9-4abc-8be2-ece9d4213d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.736908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df10337-39b9-4abc-8be2-ece9d4213d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.739142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df10337-39b9-4abc-8be2-ece9d4213d0b admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.750941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5488b51-2d28-420c-b39e-8335a15d4e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.751636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5488b51-2d28-420c-b39e-8335a15d4e79 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.751923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5488b51-2d28-420c-b39e-8335a15d4e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.753547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5488b51-2d28-420c-b39e-8335a15d4e79 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.753547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5488b51-2d28-420c-b39e-8335a15d4e79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.755284 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5488b51-2d28-420c-b39e-8335a15d4e79 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.767706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:42.780039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1092/2177] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:42 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:42.780868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df10337-39b9-4abc-8be2-ece9d4213d0b admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.786114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1086/2178] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:42 2022] POST /identity/v3/users => generated 285 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:42.788626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5488b51-2d28-420c-b39e-8335a15d4e79 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.805659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5488b51-2d28-420c-b39e-8335a15d4e79 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.886667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df10337-39b9-4abc-8be2-ece9d4213d0b admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.891842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40081e-6751-4250-9f69-38ab535c7267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.893312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.893486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40081e-6751-4250-9f69-38ab535c7267 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.893741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40081e-6751-4250-9f69-38ab535c7267 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.894030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40081e-6751-4250-9f69-38ab535c7267 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.894742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40081e-6751-4250-9f69-38ab535c7267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.895050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.895493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.895962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] PATH_INFO: `/v3/users/68612f130bb1478bb75da751791a2411` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.896910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40081e-6751-4250-9f69-38ab535c7267 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.897619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=68612f130bb1478bb75da751791a2411)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.899778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.947981 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1087/2179] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:42 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:42.949464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40081e-6751-4250-9f69-38ab535c7267 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.964114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40081e-6751-4250-9f69-38ab535c7267 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:42.971123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6bd17-661a-470f-969f-0f98d79958f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:42.972649 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6bd17-661a-470f-969f-0f98d79958f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:42.973117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6bd17-661a-470f-969f-0f98d79958f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:42.973561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6bd17-661a-470f-969f-0f98d79958f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:42.974441 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6bd17-661a-470f-969f-0f98d79958f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:42.976254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6bd17-661a-470f-969f-0f98d79958f9 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:42.991211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1088/2180] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:42.991211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6bd17-661a-470f-969f-0f98d79958f9 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:42.994062 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68612f130bb1478bb75da751791a2411'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:43.000876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6bd17-661a-470f-969f-0f98d79958f9 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:43.003220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68612f130bb1478bb75da751791a2411'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:43.004433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68612f130bb1478bb75da751791a2411'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:43.006000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ed55c-1b14-4348-a150-023aaeccba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:43.006678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ed55c-1b14-4348-a150-023aaeccba04 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:43.006961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ed55c-1b14-4348-a150-023aaeccba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:43.007240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ed55c-1b14-4348-a150-023aaeccba04 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:43.007973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ed55c-1b14-4348-a150-023aaeccba04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:43.009614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ed55c-1b14-4348-a150-023aaeccba04 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:43.026089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1089/2181] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:43.027136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ed55c-1b14-4348-a150-023aaeccba04 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:43.028898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a60ed55c-1b14-4348-a150-023aaeccba04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:43.035490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68612f130bb1478bb75da751791a2411'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:43.043981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68612f130bb1478bb75da751791a2411'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:43.048199 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ed55c-1b14-4348-a150-023aaeccba04 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:43.052951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1093/2182] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:42 2022] DELETE /identity/v3/users/68612f130bb1478bb75da751791a2411 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:43.054061 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:43.056010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:43.125584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715baaca-5880-4d97-ae5a-424e3a96c776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:43.126739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:43.127251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715baaca-5880-4d97-ae5a-424e3a96c776 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:43.128484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715baaca-5880-4d97-ae5a-424e3a96c776 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:43.128700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715baaca-5880-4d97-ae5a-424e3a96c776 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:43.129778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715baaca-5880-4d97-ae5a-424e3a96c776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:43.132350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715baaca-5880-4d97-ae5a-424e3a96c776 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:43.145868 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1090/2183] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:43.146934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75374c0-32ae-422f-9039-5bef2beac95a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:43.147229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75374c0-32ae-422f-9039-5bef2beac95a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:43.147510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75374c0-32ae-422f-9039-5bef2beac95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:43.148030 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb229494-3afd-464c-8f69-d772655f8b5c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:43.153489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f196da-35e8-4830-a8ff-d887b30a40a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:43.156225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f196da-35e8-4830-a8ff-d887b30a40a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:43.156559 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f196da-35e8-4830-a8ff-d887b30a40a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:43.156845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f196da-35e8-4830-a8ff-d887b30a40a9 admin admin] PATH_INFO: `/v3/projects/33cfa18230914e2b90e860ed75da8f0c/users/9b1b157194524c59b092dc390c589c93/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:43.172575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d75374c0-32ae-422f-9039-5bef2beac95a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:43.174409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d75374c0-32ae-422f-9039-5bef2beac95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:43.182799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f196da-35e8-4830-a8ff-d887b30a40a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cfa18230914e2b90e860ed75da8f0c, user_id=9b1b157194524c59b092dc390c589c93, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:43.188654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1091/2184] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:43.188872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f196da-35e8-4830-a8ff-d887b30a40a9 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:43.190619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75374c0-32ae-422f-9039-5bef2beac95a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:43.206778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75374c0-32ae-422f-9039-5bef2beac95a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:43.210338 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1094/2185] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:43 2022] PUT /identity/v3/projects/33cfa18230914e2b90e860ed75da8f0c/users/9b1b157194524c59b092dc390c589c93/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:43.211679 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f196da-35e8-4830-a8ff-d887b30a40a9 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:43.214868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a3f196da-35e8-4830-a8ff-d887b30a40a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:43.284402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:43.299274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3540a-6b2b-4ad6-85fd-4c63f7a1fa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:43.300106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3540a-6b2b-4ad6-85fd-4c63f7a1fa26 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:43.300550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3540a-6b2b-4ad6-85fd-4c63f7a1fa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:43.300837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3540a-6b2b-4ad6-85fd-4c63f7a1fa26 admin admin] PATH_INFO: `/v3/projects/e56267d619ec479f9b0b705174031570/users/9b3057cffda44ff0b4fe28aafb33bda9/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:43.302606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f196da-35e8-4830-a8ff-d887b30a40a9 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:43.308014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4716c2-a444-4baa-b82f-048a7a5ea672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:43.309167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4716c2-a444-4baa-b82f-048a7a5ea672 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:43.309697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4716c2-a444-4baa-b82f-048a7a5ea672 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:43.310205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4716c2-a444-4baa-b82f-048a7a5ea672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:43.320017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3540a-6b2b-4ad6-85fd-4c63f7a1fa26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56267d619ec479f9b0b705174031570, user_id=9b3057cffda44ff0b4fe28aafb33bda9, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:43.323010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4716c2-a444-4baa-b82f-048a7a5ea672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:43.324095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3540a-6b2b-4ad6-85fd-4c63f7a1fa26 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:43.325198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4716c2-a444-4baa-b82f-048a7a5ea672 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:43.339963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1095/2186] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:43.341288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4716c2-a444-4baa-b82f-048a7a5ea672 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:43.341545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1092/2187] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:43 2022] PUT /identity/v3/projects/e56267d619ec479f9b0b705174031570/users/9b3057cffda44ff0b4fe28aafb33bda9/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:43.353155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc5e2-4c88-44df-9a26-03d74a127fc8 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:43.353850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc5e2-4c88-44df-9a26-03d74a127fc8 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:43.353850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc5e2-4c88-44df-9a26-03d74a127fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:43.367843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4716c2-a444-4baa-b82f-048a7a5ea672 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:43.399825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4bfdc5e2-4c88-44df-9a26-03d74a127fc8 None None] MFA Rules not processed for user `9b3057cffda44ff0b4fe28aafb33bda9`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:43.401144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4bfdc5e2-4c88-44df-9a26-03d74a127fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:43.503145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969d54e-c721-49f2-9ded-c8226dc1b12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:43.508541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969d54e-c721-49f2-9ded-c8226dc1b12c admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:43.509355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969d54e-c721-49f2-9ded-c8226dc1b12c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:43.510000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969d54e-c721-49f2-9ded-c8226dc1b12c admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:43.511314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969d54e-c721-49f2-9ded-c8226dc1b12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:43.520319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969d54e-c721-49f2-9ded-c8226dc1b12c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:43.550996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1096/2188] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:43.567358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969d54e-c721-49f2-9ded-c8226dc1b12c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:43.590476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1093/2189] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:43.593337 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdc5e2-4c88-44df-9a26-03d74a127fc8 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:43.611268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969d54e-c721-49f2-9ded-c8226dc1b12c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:43.626696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e33cd5-e570-42ef-9ffb-326d6ac49f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:43.627870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e33cd5-e570-42ef-9ffb-326d6ac49f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:43.628022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e33cd5-e570-42ef-9ffb-326d6ac49f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:43.628273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e33cd5-e570-42ef-9ffb-326d6ac49f36 admin admin] PATH_INFO: `/v3/projects/33cfa18230914e2b90e860ed75da8f0c/users/9b1b157194524c59b092dc390c589c93/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:43.633233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e33cd5-e570-42ef-9ffb-326d6ac49f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cfa18230914e2b90e860ed75da8f0c, user_id=9b1b157194524c59b092dc390c589c93, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:43.636374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e33cd5-e570-42ef-9ffb-326d6ac49f36 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:43.657033 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdc5e2-4c88-44df-9a26-03d74a127fc8 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:43.660907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1097/2190] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:43 2022] PUT /identity/v3/projects/33cfa18230914e2b90e860ed75da8f0c/users/9b1b157194524c59b092dc390c589c93/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:43.671877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255c0de-d6ef-4c79-b971-2ab41b428bb3 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:43.672544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255c0de-d6ef-4c79-b971-2ab41b428bb3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:43.675989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255c0de-d6ef-4c79-b971-2ab41b428bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:43.756615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-6255c0de-d6ef-4c79-b971-2ab41b428bb3 None None] MFA Rules not processed for user `9b1b157194524c59b092dc390c589c93`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:43.758372 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6255c0de-d6ef-4c79-b971-2ab41b428bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:43.886039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af86c5-69a0-4417-8622-8e15ce15cfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:43.892388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af86c5-69a0-4417-8622-8e15ce15cfae admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:43.892901 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af86c5-69a0-4417-8622-8e15ce15cfae admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:43.893411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af86c5-69a0-4417-8622-8e15ce15cfae admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:43.894239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af86c5-69a0-4417-8622-8e15ce15cfae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:43.896215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:43.897121 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af86c5-69a0-4417-8622-8e15ce15cfae admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:43.958300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1098/2191] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:43.960664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255c0de-d6ef-4c79-b971-2ab41b428bb3 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:43.961607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6255c0de-d6ef-4c79-b971-2ab41b428bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:43.964193 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1094/2192] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:43 2022] POST /identity/v3/projects => generated 373 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:56:43.969720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af86c5-69a0-4417-8622-8e15ce15cfae admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:43.971236 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b8af86c5-69a0-4417-8622-8e15ce15cfae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:43.980663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255c0de-d6ef-4c79-b971-2ab41b428bb3 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:44.083005 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af86c5-69a0-4417-8622-8e15ce15cfae admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:44.089601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7071874-8a4f-490d-88bb-819be38e3bc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:44.090644 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7071874-8a4f-490d-88bb-819be38e3bc0 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:44.091433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7071874-8a4f-490d-88bb-819be38e3bc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:44.092732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7071874-8a4f-490d-88bb-819be38e3bc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:44.101219 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e7071874-8a4f-490d-88bb-819be38e3bc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:44.102268 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:44.109485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:44.110888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:44.111374 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:44.111816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] PATH_INFO: `/v3/projects/b5c3218e26bd4f30baf4943275eb35a1` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:44.113797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5c3218e26bd4f30baf4943275eb35a1)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:44.117196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:44.200446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5c3218e26bd4f30baf4943275eb35a1', 'request_id': 'req-3630de6b-da3e-440c-92d5-bb2d33a63849'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.200648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7071874-8a4f-490d-88bb-819be38e3bc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:44.202355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7071874-8a4f-490d-88bb-819be38e3bc0 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:44.217434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1095/2193] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:44.218774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7071874-8a4f-490d-88bb-819be38e3bc0 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:44.227038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5c3218e26bd4f30baf4943275eb35a1', 'request_id': 'req-3630de6b-da3e-440c-92d5-bb2d33a63849'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.234584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7071874-8a4f-490d-88bb-819be38e3bc0 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:44.253138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5c3218e26bd4f30baf4943275eb35a1', 'request_id': 'req-3630de6b-da3e-440c-92d5-bb2d33a63849'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.278114 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] The token cache is being invalidate because project b5c3218e26bd4f30baf4943275eb35a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:44.278527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.280644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.282410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1099/2194] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:43 2022] DELETE /identity/v3/projects/b5c3218e26bd4f30baf4943275eb35a1 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:44.289362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:44.290931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:44.345205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732eb12-c5a0-44b8-bf1d-00b004da3a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:44.346413 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732eb12-c5a0-44b8-bf1d-00b004da3a58 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:44.346942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732eb12-c5a0-44b8-bf1d-00b004da3a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:44.347251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732eb12-c5a0-44b8-bf1d-00b004da3a58 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:44.348460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732eb12-c5a0-44b8-bf1d-00b004da3a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:44.351322 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732eb12-c5a0-44b8-bf1d-00b004da3a58 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:44.377650 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1096/2195] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:44 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:44.378442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732eb12-c5a0-44b8-bf1d-00b004da3a58 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:44.380148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a732eb12-c5a0-44b8-bf1d-00b004da3a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:44.394934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732eb12-c5a0-44b8-bf1d-00b004da3a58 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:44.396959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630de6b-da3e-440c-92d5-bb2d33a63849 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:44.400283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:44.401417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:44.401879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:44.402209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] PATH_INFO: `/v3/users/284d7172360641a6b039d1bc23e1152e` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:44.402823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:44.403492 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=284d7172360641a6b039d1bc23e1152e)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:44.403880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:44.404304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:44.405794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] PATH_INFO: `/v3/users/bd70a233f1eb4ca08bd404919d55c9e1` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:44.406028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:44.407247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd70a233f1eb4ca08bd404919d55c9e1)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:44.409330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:44.428753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:44.476616 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '284d7172360641a6b039d1bc23e1152e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.485717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd70a233f1eb4ca08bd404919d55c9e1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.486979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '284d7172360641a6b039d1bc23e1152e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.494723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd70a233f1eb4ca08bd404919d55c9e1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.495745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd70a233f1eb4ca08bd404919d55c9e1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.500233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '284d7172360641a6b039d1bc23e1152e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.501719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '284d7172360641a6b039d1bc23e1152e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.510556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd70a233f1eb4ca08bd404919d55c9e1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.522032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '284d7172360641a6b039d1bc23e1152e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.522278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd70a233f1eb4ca08bd404919d55c9e1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:44.530426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1097/2196] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:44 2022] DELETE /identity/v3/users/284d7172360641a6b039d1bc23e1152e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:44.531330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:44.531887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1100/2197] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:44 2022] DELETE /identity/v3/users/bd70a233f1eb4ca08bd404919d55c9e1 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:44.541685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:44.556102 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:44.642301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cba18-2921-4422-9aed-a091f17c36ab admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:44.716737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56426a75-d976-434b-93a8-fcfb37ff8702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:44.717424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56426a75-d976-434b-93a8-fcfb37ff8702 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:44.718137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56426a75-d976-434b-93a8-fcfb37ff8702 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:44.718137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56426a75-d976-434b-93a8-fcfb37ff8702 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:44.718716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56426a75-d976-434b-93a8-fcfb37ff8702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:44.728978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56426a75-d976-434b-93a8-fcfb37ff8702 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:44.749110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1098/2198] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:44.750618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:44.752532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:44.768842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ceed88-9fee-459e-8318-fab663de2567 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:44.774777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267048c-a56f-4696-bac1-b80d3f379792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:44.775908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267048c-a56f-4696-bac1-b80d3f379792 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:44.776269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267048c-a56f-4696-bac1-b80d3f379792 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:44.776702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267048c-a56f-4696-bac1-b80d3f379792 admin admin] PATH_INFO: `/v3/projects/1566bbe526734c33813b5cdca65fbcfe/users/97acbfe7996c4e069e7424e52d91e11c/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:44.811987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267048c-a56f-4696-bac1-b80d3f379792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1566bbe526734c33813b5cdca65fbcfe, user_id=97acbfe7996c4e069e7424e52d91e11c, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:44.815753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267048c-a56f-4696-bac1-b80d3f379792 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:44.831812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1101/2199] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:44 2022] PUT /identity/v3/projects/1566bbe526734c33813b5cdca65fbcfe/users/97acbfe7996c4e069e7424e52d91e11c/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:44.836755 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56426a75-d976-434b-93a8-fcfb37ff8702 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:44.849749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56426a75-d976-434b-93a8-fcfb37ff8702 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:44.922332 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaefe22-71cb-499a-b644-99c6ce6ea8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:44.922594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaefe22-71cb-499a-b644-99c6ce6ea8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:44.922779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaefe22-71cb-499a-b644-99c6ce6ea8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:44.923066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaefe22-71cb-499a-b644-99c6ce6ea8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:44.923771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaefe22-71cb-499a-b644-99c6ce6ea8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:44.925480 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaefe22-71cb-499a-b644-99c6ce6ea8e5 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:44.940636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1099/2200] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:44.941873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267048c-a56f-4696-bac1-b80d3f379792 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:44.953700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267048c-a56f-4696-bac1-b80d3f379792 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:44.958581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d85618-34b7-458e-ba1c-17a3ca49066f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:44.959561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d85618-34b7-458e-ba1c-17a3ca49066f admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:44.959943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d85618-34b7-458e-ba1c-17a3ca49066f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:44.960295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d85618-34b7-458e-ba1c-17a3ca49066f admin admin] PATH_INFO: `/v3/projects/1566bbe526734c33813b5cdca65fbcfe/users/97acbfe7996c4e069e7424e52d91e11c/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:44.962468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d85618-34b7-458e-ba1c-17a3ca49066f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1566bbe526734c33813b5cdca65fbcfe, user_id=97acbfe7996c4e069e7424e52d91e11c, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:44.965280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d85618-34b7-458e-ba1c-17a3ca49066f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:44.977395 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1102/2201] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:44 2022] PUT /identity/v3/projects/1566bbe526734c33813b5cdca65fbcfe/users/97acbfe7996c4e069e7424e52d91e11c/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:44.988219 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052deb2-5fa6-4dc0-be55-c96e96185bfc None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:44.988568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052deb2-5fa6-4dc0-be55-c96e96185bfc None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:44.988916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052deb2-5fa6-4dc0-be55-c96e96185bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.033227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-b052deb2-5fa6-4dc0-be55-c96e96185bfc None None] MFA Rules not processed for user `97acbfe7996c4e069e7424e52d91e11c`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:45.034662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b052deb2-5fa6-4dc0-be55-c96e96185bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:45.079833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:45.099530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d85618-34b7-458e-ba1c-17a3ca49066f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.101427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-33d85618-34b7-458e-ba1c-17a3ca49066f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:45.133231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1100/2202] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:45.141070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052deb2-5fa6-4dc0-be55-c96e96185bfc None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.142522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b052deb2-5fa6-4dc0-be55-c96e96185bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:45.233368 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052deb2-5fa6-4dc0-be55-c96e96185bfc None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:45.237801 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd141b-4c57-4625-a0d3-67a36e8d9943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:45.238475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd141b-4c57-4625-a0d3-67a36e8d9943 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.238603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d85618-34b7-458e-ba1c-17a3ca49066f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:45.238762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd141b-4c57-4625-a0d3-67a36e8d9943 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.239045 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd141b-4c57-4625-a0d3-67a36e8d9943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.240964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-01cd141b-4c57-4625-a0d3-67a36e8d9943 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:45.245702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:45.247082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.248033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.248033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] PATH_INFO: `/v3/projects/79723635097c40588d8dfedceb79ee68` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.250654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=79723635097c40588d8dfedceb79ee68)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:45.252761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:45.315103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79723635097c40588d8dfedceb79ee68', 'request_id': 'req-2200faac-a595-44ee-993c-db1e0d1e51cc'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:45.320217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:45.325423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79723635097c40588d8dfedceb79ee68', 'request_id': 'req-2200faac-a595-44ee-993c-db1e0d1e51cc'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:45.327232 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd141b-4c57-4625-a0d3-67a36e8d9943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:45.328969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd141b-4c57-4625-a0d3-67a36e8d9943 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:45.336103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79723635097c40588d8dfedceb79ee68', 'request_id': 'req-2200faac-a595-44ee-993c-db1e0d1e51cc'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:45.347687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1101/2203] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:45 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:56:45.348065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] The token cache is being invalidate because project 79723635097c40588d8dfedceb79ee68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:45.348161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:45.348703 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd141b-4c57-4625-a0d3-67a36e8d9943 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.348844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2200faac-a595-44ee-993c-db1e0d1e51cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:45.350088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-01cd141b-4c57-4625-a0d3-67a36e8d9943 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:45.354724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1103/2204] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:45 2022] DELETE /identity/v3/projects/79723635097c40588d8dfedceb79ee68 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:45.356167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683a25-08f7-432c-b1b9-884b1d3a26c1 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.356495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683a25-08f7-432c-b1b9-884b1d3a26c1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.356774 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683a25-08f7-432c-b1b9-884b1d3a26c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.365889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd141b-4c57-4625-a0d3-67a36e8d9943 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:45.377789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ff683a25-08f7-432c-b1b9-884b1d3a26c1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:45.379074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ff683a25-08f7-432c-b1b9-884b1d3a26c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:45.437081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:45.438015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.438304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.438583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] PATH_INFO: `/v3/projects/d03dc47dd1fc4766a123e56efcbdffaf` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.440188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d03dc47dd1fc4766a123e56efcbdffaf)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:45.442253 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:45.455517 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1104/2205] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:45.463462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430733-6e2d-45cd-890f-7e1ea93d3394 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.464299 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430733-6e2d-45cd-890f-7e1ea93d3394 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.464788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430733-6e2d-45cd-890f-7e1ea93d3394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.487812 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:45.492112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-2e430733-6e2d-45cd-890f-7e1ea93d3394 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:45.493289 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2e430733-6e2d-45cd-890f-7e1ea93d3394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:45.514940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd03dc47dd1fc4766a123e56efcbdffaf', 'request_id': 'req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:45.526520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd03dc47dd1fc4766a123e56efcbdffaf', 'request_id': 'req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:45.536666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd03dc47dd1fc4766a123e56efcbdffaf', 'request_id': 'req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:45.551174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] The token cache is being invalidate because project d03dc47dd1fc4766a123e56efcbdffaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:45.551383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:45.552460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a22a4f51-b036-43a6-8f00-2bbdd8d0bf87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:45.554593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1102/2206] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:45 2022] DELETE /identity/v3/projects/d03dc47dd1fc4766a123e56efcbdffaf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:45.560247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e79c8-53a4-4592-baca-548242634928 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.560990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e79c8-53a4-4592-baca-548242634928 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.561227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e79c8-53a4-4592-baca-548242634928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.578043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1105/2207] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:45.582893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e430733-6e2d-45cd-890f-7e1ea93d3394 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.587619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-8c2e79c8-53a4-4592-baca-548242634928 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:45.589342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8c2e79c8-53a4-4592-baca-548242634928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:45.595234 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e430733-6e2d-45cd-890f-7e1ea93d3394 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:45.599201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1103/2208] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:45.602102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c3518-8bc9-4bac-be95-be1f4b88fb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:45.602779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c3518-8bc9-4bac-be95-be1f4b88fb32 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.603065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c3518-8bc9-4bac-be95-be1f4b88fb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.604789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c3518-8bc9-4bac-be95-be1f4b88fb32 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.606115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c3518-8bc9-4bac-be95-be1f4b88fb32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:45.606301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c3518-8bc9-4bac-be95-be1f4b88fb32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:45.606955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4004f-39d9-48f8-adc7-0f0557ade194 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.607280 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4004f-39d9-48f8-adc7-0f0557ade194 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.608095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4004f-39d9-48f8-adc7-0f0557ade194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.608544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c3518-8bc9-4bac-be95-be1f4b88fb32 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:45.624420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1106/2209] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:56:45.629522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c3518-8bc9-4bac-be95-be1f4b88fb32 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.629687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-fbb4004f-39d9-48f8-adc7-0f0557ade194 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:45.631142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fbb4004f-39d9-48f8-adc7-0f0557ade194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:45.641643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c3518-8bc9-4bac-be95-be1f4b88fb32 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:45.642182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1104/2210] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:45.646859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4004f-39d9-48f8-adc7-0f0557ade194 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.650591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0afa2-0736-46a1-84cd-fc52a16db3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:45.652062 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0afa2-0736-46a1-84cd-fc52a16db3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.652726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0afa2-0736-46a1-84cd-fc52a16db3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.653016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0afa2-0736-46a1-84cd-fc52a16db3f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.653742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0afa2-0736-46a1-84cd-fc52a16db3f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:45.656525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0afa2-0736-46a1-84cd-fc52a16db3f5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:45.657560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4004f-39d9-48f8-adc7-0f0557ade194 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:45.664563 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5fdcd-56fe-4877-a88f-de886630a887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:45.665294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5fdcd-56fe-4877-a88f-de886630a887 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.665650 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5fdcd-56fe-4877-a88f-de886630a887 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.665940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5fdcd-56fe-4877-a88f-de886630a887 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.666577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5fdcd-56fe-4877-a88f-de886630a887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:45.666886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5fdcd-56fe-4877-a88f-de886630a887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:45.669406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5fdcd-56fe-4877-a88f-de886630a887 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:45.688532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1107/2211] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:45 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:45.694448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1105/2212] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:56:45.698715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0afa2-0736-46a1-84cd-fc52a16db3f5 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.700039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5fdcd-56fe-4877-a88f-de886630a887 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.711294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5fdcd-56fe-4877-a88f-de886630a887 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:45.783664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:45.802136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0afa2-0736-46a1-84cd-fc52a16db3f5 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:45.802817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cb0a3-f9e7-4cc8-9e20-c7023a24ee42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:45.803981 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cb0a3-f9e7-4cc8-9e20-c7023a24ee42 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.804635 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cb0a3-f9e7-4cc8-9e20-c7023a24ee42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.804915 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cb0a3-f9e7-4cc8-9e20-c7023a24ee42 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.805782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cb0a3-f9e7-4cc8-9e20-c7023a24ee42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:45.809952 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cb0a3-f9e7-4cc8-9e20-c7023a24ee42 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:45.811570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fd457-69ee-4170-9e15-0e7ef13f6f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:45.812394 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fd457-69ee-4170-9e15-0e7ef13f6f4b admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.812730 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fd457-69ee-4170-9e15-0e7ef13f6f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.813017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fd457-69ee-4170-9e15-0e7ef13f6f4b admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.813719 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fd457-69ee-4170-9e15-0e7ef13f6f4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:45.815966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fd457-69ee-4170-9e15-0e7ef13f6f4b admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:45.843772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1108/2213] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:45 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:45.845329 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fd457-69ee-4170-9e15-0e7ef13f6f4b admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.865887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1106/2214] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:45 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:45.866027 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fd457-69ee-4170-9e15-0e7ef13f6f4b admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:45.868151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cb0a3-f9e7-4cc8-9e20-c7023a24ee42 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.932610 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03f51-73d5-40c9-94ce-76ab77e6f418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:45.933300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03f51-73d5-40c9-94ce-76ab77e6f418 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.933590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03f51-73d5-40c9-94ce-76ab77e6f418 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.933872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03f51-73d5-40c9-94ce-76ab77e6f418 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.934616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b03f51-73d5-40c9-94ce-76ab77e6f418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:45.936280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b03f51-73d5-40c9-94ce-76ab77e6f418 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:45.954313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1109/2215] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:45.955292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cb0a3-f9e7-4cc8-9e20-c7023a24ee42 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:45.956885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03f51-73d5-40c9-94ce-76ab77e6f418 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.960203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895decf5-4f90-4b52-9468-d4ed92ddabaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:45.960927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895decf5-4f90-4b52-9468-d4ed92ddabaa admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.961219 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895decf5-4f90-4b52-9468-d4ed92ddabaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.961501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895decf5-4f90-4b52-9468-d4ed92ddabaa admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.962226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895decf5-4f90-4b52-9468-d4ed92ddabaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:45.964175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895decf5-4f90-4b52-9468-d4ed92ddabaa admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:45.969818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03f51-73d5-40c9-94ce-76ab77e6f418 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:45.977163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44c5b6-0c08-40ed-b827-4c4bde275d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:45.978045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44c5b6-0c08-40ed-b827-4c4bde275d27 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:45.978346 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44c5b6-0c08-40ed-b827-4c4bde275d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:45.978695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44c5b6-0c08-40ed-b827-4c4bde275d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:45.979510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44c5b6-0c08-40ed-b827-4c4bde275d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:45.981233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44c5b6-0c08-40ed-b827-4c4bde275d27 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:45.987400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1107/2216] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:45 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:56:45.991562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895decf5-4f90-4b52-9468-d4ed92ddabaa admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.996533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1110/2217] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:45.997542 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44c5b6-0c08-40ed-b827-4c4bde275d27 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:45.998367 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:46.003309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895decf5-4f90-4b52-9468-d4ed92ddabaa admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:46.007991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44c5b6-0c08-40ed-b827-4c4bde275d27 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:46.010686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d85ac2-69d6-48ef-805b-127f8806874d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:46.011352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85ac2-69d6-48ef-805b-127f8806874d admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.011634 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85ac2-69d6-48ef-805b-127f8806874d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.011908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d85ac2-69d6-48ef-805b-127f8806874d admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.012673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d85ac2-69d6-48ef-805b-127f8806874d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:46.013015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aed615-5f5d-44fa-8d2c-e02b04a68dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:46.014404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aed615-5f5d-44fa-8d2c-e02b04a68dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.014684 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aed615-5f5d-44fa-8d2c-e02b04a68dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.014889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d85ac2-69d6-48ef-805b-127f8806874d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:46.014994 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aed615-5f5d-44fa-8d2c-e02b04a68dd6 admin admin] PATH_INFO: `/v3/projects/d5af714b14ba471980868293588841fd/users/ac1b24e50c894dcf8e887275fd91a174/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.025451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1108/2218] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:46.030076 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d85ac2-69d6-48ef-805b-127f8806874d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:46.032008 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aed615-5f5d-44fa-8d2c-e02b04a68dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5af714b14ba471980868293588841fd, user_id=ac1b24e50c894dcf8e887275fd91a174, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:46.034731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aed615-5f5d-44fa-8d2c-e02b04a68dd6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:46.041353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d85ac2-69d6-48ef-805b-127f8806874d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:46.047479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9be62-598c-4404-ae7b-0da4b24b523e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:46.048168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9be62-598c-4404-ae7b-0da4b24b523e admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.048494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9be62-598c-4404-ae7b-0da4b24b523e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.048773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9be62-598c-4404-ae7b-0da4b24b523e admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.049515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9be62-598c-4404-ae7b-0da4b24b523e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:46.049893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1111/2219] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:45 2022] PUT /identity/v3/projects/d5af714b14ba471980868293588841fd/users/ac1b24e50c894dcf8e887275fd91a174/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:46.051340 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9be62-598c-4404-ae7b-0da4b24b523e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:46.055378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836d82e-b156-4383-9926-6b6d28b09c05 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.055591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836d82e-b156-4383-9926-6b6d28b09c05 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.055890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836d82e-b156-4383-9926-6b6d28b09c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.063163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1109/2220] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:46.067536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9be62-598c-4404-ae7b-0da4b24b523e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:46.074154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c836d82e-b156-4383-9926-6b6d28b09c05 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:46.075288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c836d82e-b156-4383-9926-6b6d28b09c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:46.079104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9be62-598c-4404-ae7b-0da4b24b523e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:46.137018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1112/2221] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:46.139854 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629298e-f712-4a10-ae55-301e4d3e7ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:46.140792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629298e-f712-4a10-ae55-301e4d3e7ad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.141537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629298e-f712-4a10-ae55-301e4d3e7ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.141537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629298e-f712-4a10-ae55-301e4d3e7ad8 admin admin] PATH_INFO: `/v3/projects/8a61f17e0fed4444993f491a59305c3d/users/cc69b4321b1b42fcb2d847fd2bab6d8b/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.147332 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836d82e-b156-4383-9926-6b6d28b09c05 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:46.148832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c836d82e-b156-4383-9926-6b6d28b09c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:46.157628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629298e-f712-4a10-ae55-301e4d3e7ad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a61f17e0fed4444993f491a59305c3d, user_id=cc69b4321b1b42fcb2d847fd2bab6d8b, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:46.160418 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629298e-f712-4a10-ae55-301e4d3e7ad8 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:46.174489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1110/2222] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:46 2022] PUT /identity/v3/projects/8a61f17e0fed4444993f491a59305c3d/users/cc69b4321b1b42fcb2d847fd2bab6d8b/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:46.184809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d132bbc-8354-4bb4-9405-e72c09000e36 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.185174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d132bbc-8354-4bb4-9405-e72c09000e36 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.185913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d132bbc-8354-4bb4-9405-e72c09000e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.196058 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:46.218054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-7d132bbc-8354-4bb4-9405-e72c09000e36 None None] MFA Rules not processed for user `cc69b4321b1b42fcb2d847fd2bab6d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:46.219509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7d132bbc-8354-4bb4-9405-e72c09000e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:46.222354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836d82e-b156-4383-9926-6b6d28b09c05 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:46.276405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf218b4-77b3-42dc-bd9e-c135ec43a05b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:46.277067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf218b4-77b3-42dc-bd9e-c135ec43a05b service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.277358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf218b4-77b3-42dc-bd9e-c135ec43a05b service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.277640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf218b4-77b3-42dc-bd9e-c135ec43a05b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.290637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1111/2223] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:46.295049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d132bbc-8354-4bb4-9405-e72c09000e36 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:46.345710 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf218b4-77b3-42dc-bd9e-c135ec43a05b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:46.347365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf218b4-77b3-42dc-bd9e-c135ec43a05b service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:46.358898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:46.360583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1113/2224] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:56:46.366355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d132bbc-8354-4bb4-9405-e72c09000e36 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:46.373786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95956b-9af4-40f1-8173-dc8f9d7d5840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:46.374465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95956b-9af4-40f1-8173-dc8f9d7d5840 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.375003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95956b-9af4-40f1-8173-dc8f9d7d5840 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.375003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95956b-9af4-40f1-8173-dc8f9d7d5840 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.375709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95956b-9af4-40f1-8173-dc8f9d7d5840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:46.378106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95956b-9af4-40f1-8173-dc8f9d7d5840 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:46.412769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1112/2225] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:46 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:56:46.416842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf218b4-77b3-42dc-bd9e-c135ec43a05b service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:46.426892 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf218b4-77b3-42dc-bd9e-c135ec43a05b service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:46.490931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5bac6-d408-4800-be41-fb53015fb59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:46.491648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5bac6-d408-4800-be41-fb53015fb59e admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.491938 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5bac6-d408-4800-be41-fb53015fb59e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.492218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5bac6-d408-4800-be41-fb53015fb59e admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.492966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5bac6-d408-4800-be41-fb53015fb59e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:46.494906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5bac6-d408-4800-be41-fb53015fb59e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:46.518362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1114/2226] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:46 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:46.522598 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95956b-9af4-40f1-8173-dc8f9d7d5840 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:46.534036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95956b-9af4-40f1-8173-dc8f9d7d5840 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:46.540977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09cc8b-0a83-4ab3-a7ad-1dff80b09927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:46.541691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09cc8b-0a83-4ab3-a7ad-1dff80b09927 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.542010 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09cc8b-0a83-4ab3-a7ad-1dff80b09927 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.542444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09cc8b-0a83-4ab3-a7ad-1dff80b09927 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.544166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09cc8b-0a83-4ab3-a7ad-1dff80b09927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:46.545565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09cc8b-0a83-4ab3-a7ad-1dff80b09927 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:46.558926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1113/2227] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:46.564244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5bac6-d408-4800-be41-fb53015fb59e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:46.574881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5bac6-d408-4800-be41-fb53015fb59e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:46.579659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a6e44-ce97-487a-bf36-2313ed3270b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:46.580584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a6e44-ce97-487a-bf36-2313ed3270b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.580871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a6e44-ce97-487a-bf36-2313ed3270b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.581152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a6e44-ce97-487a-bf36-2313ed3270b6 admin admin] PATH_INFO: `/v3/projects/da58adaadc344cabac9fba9d3155f8c5/users/44fd27491fb543a48ed8afbd847c5c4d/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.597693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a6e44-ce97-487a-bf36-2313ed3270b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da58adaadc344cabac9fba9d3155f8c5, user_id=44fd27491fb543a48ed8afbd847c5c4d, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:46.600264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a6e44-ce97-487a-bf36-2313ed3270b6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:46.617331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1115/2228] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:46 2022] PUT /identity/v3/projects/da58adaadc344cabac9fba9d3155f8c5/users/44fd27491fb543a48ed8afbd847c5c4d/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:46.621323 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09cc8b-0a83-4ab3-a7ad-1dff80b09927 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:46.632572 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09cc8b-0a83-4ab3-a7ad-1dff80b09927 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:46.699882 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b3842-cfa1-43c1-9aaa-29a2950af64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:46.700606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3842-cfa1-43c1-9aaa-29a2950af64c admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.700897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3842-cfa1-43c1-9aaa-29a2950af64c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.701233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3842-cfa1-43c1-9aaa-29a2950af64c admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.701978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b3842-cfa1-43c1-9aaa-29a2950af64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:46.703619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b3842-cfa1-43c1-9aaa-29a2950af64c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:46.716366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1114/2229] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:46.720562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a6e44-ce97-487a-bf36-2313ed3270b6 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:46.734390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a6e44-ce97-487a-bf36-2313ed3270b6 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:46.741755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3292e7-d17a-4909-96ec-a7522632d6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:46.742673 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3292e7-d17a-4909-96ec-a7522632d6d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.742958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3292e7-d17a-4909-96ec-a7522632d6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.743235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3292e7-d17a-4909-96ec-a7522632d6d1 admin admin] PATH_INFO: `/v3/projects/da58adaadc344cabac9fba9d3155f8c5/users/44fd27491fb543a48ed8afbd847c5c4d/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.745516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3292e7-d17a-4909-96ec-a7522632d6d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da58adaadc344cabac9fba9d3155f8c5, user_id=44fd27491fb543a48ed8afbd847c5c4d, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:46.750536 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3292e7-d17a-4909-96ec-a7522632d6d1 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:46.765422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:46.776831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1116/2230] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:46 2022] PUT /identity/v3/projects/da58adaadc344cabac9fba9d3155f8c5/users/44fd27491fb543a48ed8afbd847c5c4d/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:46.788941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871c064-66b3-4fdb-9817-d10dd92f377b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:46.789137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871c064-66b3-4fdb-9817-d10dd92f377b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:46.789691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871c064-66b3-4fdb-9817-d10dd92f377b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:46.834905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-0871c064-66b3-4fdb-9817-d10dd92f377b None None] MFA Rules not processed for user `44fd27491fb543a48ed8afbd847c5c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:46.836231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0871c064-66b3-4fdb-9817-d10dd92f377b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:46.923283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1115/2231] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:46.934668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3292e7-d17a-4909-96ec-a7522632d6d1 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:46.936704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ad3292e7-d17a-4909-96ec-a7522632d6d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:47.001464 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3292e7-d17a-4909-96ec-a7522632d6d1 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:47.005837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262907c5-ecca-4875-a640-eb32f6169244 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:47.006576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262907c5-ecca-4875-a640-eb32f6169244 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:47.006873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262907c5-ecca-4875-a640-eb32f6169244 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:47.007175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262907c5-ecca-4875-a640-eb32f6169244 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:47.022019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262907c5-ecca-4875-a640-eb32f6169244 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:47.023639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262907c5-ecca-4875-a640-eb32f6169244 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:47.035750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1117/2232] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:47.108184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871c064-66b3-4fdb-9817-d10dd92f377b None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:47.180166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:47.200028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871c064-66b3-4fdb-9817-d10dd92f377b None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:47.205880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:47.206927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:47.207356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:47.207642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] PATH_INFO: `/v3/users/cc69b4321b1b42fcb2d847fd2bab6d8b` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:47.209834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc69b4321b1b42fcb2d847fd2bab6d8b)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:47.212049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:47.267019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc69b4321b1b42fcb2d847fd2bab6d8b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:47.277373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc69b4321b1b42fcb2d847fd2bab6d8b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:47.286502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc69b4321b1b42fcb2d847fd2bab6d8b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:47.287308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc69b4321b1b42fcb2d847fd2bab6d8b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:47.302802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc69b4321b1b42fcb2d847fd2bab6d8b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:47.311493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1116/2233] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:47 2022] DELETE /identity/v3/users/cc69b4321b1b42fcb2d847fd2bab6d8b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:47.316384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83711b4c-7200-45d1-97c9-dccc62f08921 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:47.316384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83711b4c-7200-45d1-97c9-dccc62f08921 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:47.316943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83711b4c-7200-45d1-97c9-dccc62f08921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:47.336524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-83711b4c-7200-45d1-97c9-dccc62f08921 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:47.337776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-83711b4c-7200-45d1-97c9-dccc62f08921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:47.422771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1118/2234] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:47.430465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:47.433280 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:47.509055 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20215bf-a563-47c3-8cf6-65330606980f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:47.515715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ae7d9-c7e0-4fde-b0e5-f3b058f01bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:47.516423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae7d9-c7e0-4fde-b0e5-f3b058f01bac service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:47.516710 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae7d9-c7e0-4fde-b0e5-f3b058f01bac service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:47.516989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae7d9-c7e0-4fde-b0e5-f3b058f01bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:47.528475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ae7d9-c7e0-4fde-b0e5-f3b058f01bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:47.530105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ae7d9-c7e0-4fde-b0e5-f3b058f01bac service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:47.541148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1117/2235] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:47.881139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83711b4c-7200-45d1-97c9-dccc62f08921 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:47.882939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-83711b4c-7200-45d1-97c9-dccc62f08921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:47.906677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83711b4c-7200-45d1-97c9-dccc62f08921 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:47.914332 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:47.923930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:47.923930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:47.923930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] PATH_INFO: `/v3/projects/8a61f17e0fed4444993f491a59305c3d` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:47.932545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a61f17e0fed4444993f491a59305c3d)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:47.937481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:48.023187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a61f17e0fed4444993f491a59305c3d', 'request_id': 'req-adc7a380-55f7-4b0e-8126-832d4ab82bab'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.040170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a61f17e0fed4444993f491a59305c3d', 'request_id': 'req-adc7a380-55f7-4b0e-8126-832d4ab82bab'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.050751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a61f17e0fed4444993f491a59305c3d', 'request_id': 'req-adc7a380-55f7-4b0e-8126-832d4ab82bab'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.069784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] The token cache is being invalidate because project 8a61f17e0fed4444993f491a59305c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:48.070016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.070711 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.072956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1119/2236] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:47 2022] DELETE /identity/v3/projects/8a61f17e0fed4444993f491a59305c3d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:48.077266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ae7d9-c7e0-4fde-b0e5-f3b058f01bac service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:48.087547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ae7d9-c7e0-4fde-b0e5-f3b058f01bac service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:48.158064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb40dcf-f472-47d1-ba51-37266fb7866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:48.159007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb40dcf-f472-47d1-ba51-37266fb7866f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:48.159567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb40dcf-f472-47d1-ba51-37266fb7866f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:48.159685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb40dcf-f472-47d1-ba51-37266fb7866f admin admin] PATH_INFO: `/v3/users/44fd27491fb543a48ed8afbd847c5c4d` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:48.161052 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb40dcf-f472-47d1-ba51-37266fb7866f admin admin] RBAC: Authorizing `identity:delete_user(user_id=44fd27491fb543a48ed8afbd847c5c4d)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:48.163076 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb40dcf-f472-47d1-ba51-37266fb7866f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:48.180646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:48.241836 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4eb40dcf-f472-47d1-ba51-37266fb7866f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44fd27491fb543a48ed8afbd847c5c4d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.252577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4eb40dcf-f472-47d1-ba51-37266fb7866f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44fd27491fb543a48ed8afbd847c5c4d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.265420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4eb40dcf-f472-47d1-ba51-37266fb7866f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44fd27491fb543a48ed8afbd847c5c4d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.266265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4eb40dcf-f472-47d1-ba51-37266fb7866f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44fd27491fb543a48ed8afbd847c5c4d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.289247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4eb40dcf-f472-47d1-ba51-37266fb7866f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44fd27491fb543a48ed8afbd847c5c4d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.296389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1118/2237] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:48 2022] DELETE /identity/v3/users/44fd27491fb543a48ed8afbd847c5c4d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:48.664117 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:48.665696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:48.681152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7a380-55f7-4b0e-8126-832d4ab82bab admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:48.710724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:48.764034 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:48.765327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:48.765670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:48.766175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] PATH_INFO: `/v3/projects/da58adaadc344cabac9fba9d3155f8c5` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:48.768229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da58adaadc344cabac9fba9d3155f8c5)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:48.771656 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:48.839531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da58adaadc344cabac9fba9d3155f8c5', 'request_id': 'req-20474466-ecd7-497b-9e52-6315e1b219c1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.850236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da58adaadc344cabac9fba9d3155f8c5', 'request_id': 'req-20474466-ecd7-497b-9e52-6315e1b219c1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.858172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da58adaadc344cabac9fba9d3155f8c5', 'request_id': 'req-20474466-ecd7-497b-9e52-6315e1b219c1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.873621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] The token cache is being invalidate because project da58adaadc344cabac9fba9d3155f8c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:48.873926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.874708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-20474466-ecd7-497b-9e52-6315e1b219c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:48.876687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1120/2238] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:48 2022] DELETE /identity/v3/projects/da58adaadc344cabac9fba9d3155f8c5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:48.882556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eccac-1adb-472c-a4a7-27e06bd041fb None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:48.883071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eccac-1adb-472c-a4a7-27e06bd041fb None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:48.883161 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386eccac-1adb-472c-a4a7-27e06bd041fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:48.905688 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-386eccac-1adb-472c-a4a7-27e06bd041fb None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:48.907008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-386eccac-1adb-472c-a4a7-27e06bd041fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:49.022574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1119/2239] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:49.030220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a27b-c07c-402b-8e38-d47ed86153e3 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:49.030768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a27b-c07c-402b-8e38-d47ed86153e3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:49.030864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1a27b-c07c-402b-8e38-d47ed86153e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:49.052175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-d5e1a27b-c07c-402b-8e38-d47ed86153e3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:49.053458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d5e1a27b-c07c-402b-8e38-d47ed86153e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:49.063188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1121/2240] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:49.071082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386eccac-1adb-472c-a4a7-27e06bd041fb None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:49.082962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386eccac-1adb-472c-a4a7-27e06bd041fb None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:49.086789 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb48971-1d9f-42f7-a218-3411953f755c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:49.087460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48971-1d9f-42f7-a218-3411953f755c admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:49.087746 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48971-1d9f-42f7-a218-3411953f755c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:49.088025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48971-1d9f-42f7-a218-3411953f755c admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:49.088887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb48971-1d9f-42f7-a218-3411953f755c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:49.089206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb48971-1d9f-42f7-a218-3411953f755c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:49.091047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb48971-1d9f-42f7-a218-3411953f755c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:49.114842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1120/2241] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:56:49.115039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1a27b-c07c-402b-8e38-d47ed86153e3 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:49.128721 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1a27b-c07c-402b-8e38-d47ed86153e3 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:49.135382 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8348f0-0f39-4e14-84c4-68691e8aa6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:49.136156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8348f0-0f39-4e14-84c4-68691e8aa6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:49.136363 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8348f0-0f39-4e14-84c4-68691e8aa6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:49.136643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8348f0-0f39-4e14-84c4-68691e8aa6e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:49.137359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8348f0-0f39-4e14-84c4-68691e8aa6e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:49.139279 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8348f0-0f39-4e14-84c4-68691e8aa6e6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:49.183968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1122/2242] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:49 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:49.184682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb48971-1d9f-42f7-a218-3411953f755c admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:49.211994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb48971-1d9f-42f7-a218-3411953f755c admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:49.291070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf8991-6397-40c5-bafa-26f4061970be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:49.291974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf8991-6397-40c5-bafa-26f4061970be admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:49.292317 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf8991-6397-40c5-bafa-26f4061970be admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:49.292790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf8991-6397-40c5-bafa-26f4061970be admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:49.293494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf8991-6397-40c5-bafa-26f4061970be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:49.296711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf8991-6397-40c5-bafa-26f4061970be admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:49.324047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1121/2243] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:49 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:49.327920 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8348f0-0f39-4e14-84c4-68691e8aa6e6 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:49.344077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8348f0-0f39-4e14-84c4-68691e8aa6e6 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:49.349560 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35e107-b273-4707-84c1-0467ec3100af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:49.350441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35e107-b273-4707-84c1-0467ec3100af admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:49.350881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35e107-b273-4707-84c1-0467ec3100af admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:49.351514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35e107-b273-4707-84c1-0467ec3100af admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:49.352439 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35e107-b273-4707-84c1-0467ec3100af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:49.354294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35e107-b273-4707-84c1-0467ec3100af admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:49.369392 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1123/2244] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:49.373146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf8991-6397-40c5-bafa-26f4061970be admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:49.399708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf8991-6397-40c5-bafa-26f4061970be admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:49.414438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0db22-3294-4285-8255-cb26536ecfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:49.423318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0db22-3294-4285-8255-cb26536ecfc3 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:49.423670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0db22-3294-4285-8255-cb26536ecfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:49.423957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0db22-3294-4285-8255-cb26536ecfc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:49.426948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0db22-3294-4285-8255-cb26536ecfc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:49.429844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0db22-3294-4285-8255-cb26536ecfc3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:49.433982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:49.464994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1122/2245] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:49.473866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35e107-b273-4707-84c1-0467ec3100af admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:49.493119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35e107-b273-4707-84c1-0467ec3100af admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:49.508440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaca7d-760a-40b8-b3d5-bad9717e5fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:49.509545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaca7d-760a-40b8-b3d5-bad9717e5fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:49.510002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaca7d-760a-40b8-b3d5-bad9717e5fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:49.510422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaca7d-760a-40b8-b3d5-bad9717e5fc1 admin admin] PATH_INFO: `/v3/projects/f1930fe918b845209bc20c39f61de60a/users/70cdedad5b9f420693ecb1845e380c6a/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:49.545419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aaca7d-760a-40b8-b3d5-bad9717e5fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1930fe918b845209bc20c39f61de60a, user_id=70cdedad5b9f420693ecb1845e380c6a, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:49.548220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aaca7d-760a-40b8-b3d5-bad9717e5fc1 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:49.577070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1124/2246] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:49 2022] PUT /identity/v3/projects/f1930fe918b845209bc20c39f61de60a/users/70cdedad5b9f420693ecb1845e380c6a/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:49.587753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba747c3-5589-4910-bd31-03151fd60d87 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:49.588142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba747c3-5589-4910-bd31-03151fd60d87 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:49.588311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba747c3-5589-4910-bd31-03151fd60d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:49.646427 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-dba747c3-5589-4910-bd31-03151fd60d87 None None] MFA Rules not processed for user `70cdedad5b9f420693ecb1845e380c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:49.648298 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dba747c3-5589-4910-bd31-03151fd60d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:49.789217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1123/2247] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:49.802201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaca7d-760a-40b8-b3d5-bad9717e5fc1 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:49.805205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-91aaca7d-760a-40b8-b3d5-bad9717e5fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:49.894155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaca7d-760a-40b8-b3d5-bad9717e5fc1 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:49.898210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0d8f9-c0f2-4883-9792-2721748fe262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:49.898880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0d8f9-c0f2-4883-9792-2721748fe262 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:49.899223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0d8f9-c0f2-4883-9792-2721748fe262 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:49.899629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0d8f9-c0f2-4883-9792-2721748fe262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:49.906208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:49.912158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0d8f9-c0f2-4883-9792-2721748fe262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:49.914165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0d8f9-c0f2-4883-9792-2721748fe262 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:49.928248 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1125/2248] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:51.341307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24889815-d275-4204-bc67-ea6df3b0db41 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:51.341905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24889815-d275-4204-bc67-ea6df3b0db41 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:51.342483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24889815-d275-4204-bc67-ea6df3b0db41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:51.381850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-24889815-d275-4204-bc67-ea6df3b0db41 None None] MFA Rules not processed for user `ac1b24e50c894dcf8e887275fd91a174`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:51.383002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-24889815-d275-4204-bc67-ea6df3b0db41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:51.530514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1124/2249] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:51.543686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caef96d-d73a-40ad-bc07-20d1d9352913 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:51.544198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caef96d-d73a-40ad-bc07-20d1d9352913 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:51.544277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caef96d-d73a-40ad-bc07-20d1d9352913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:51.564305 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-4caef96d-d73a-40ad-bc07-20d1d9352913 None None] MFA Rules not processed for user `ac1b24e50c894dcf8e887275fd91a174`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:51.565961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4caef96d-d73a-40ad-bc07-20d1d9352913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:51.574281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1126/2250] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:51.588302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24889815-d275-4204-bc67-ea6df3b0db41 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:51.590212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-24889815-d275-4204-bc67-ea6df3b0db41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:51.699163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24889815-d275-4204-bc67-ea6df3b0db41 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:51.715348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006aa2c-5f57-4ea8-9024-9532da5e6c67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:51.716056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006aa2c-5f57-4ea8-9024-9532da5e6c67 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:51.716458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006aa2c-5f57-4ea8-9024-9532da5e6c67 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:51.717233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006aa2c-5f57-4ea8-9024-9532da5e6c67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:51.729437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006aa2c-5f57-4ea8-9024-9532da5e6c67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:51.731852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006aa2c-5f57-4ea8-9024-9532da5e6c67 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:51.743970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1125/2251] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:52.483735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caef96d-d73a-40ad-bc07-20d1d9352913 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:52.485526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4caef96d-d73a-40ad-bc07-20d1d9352913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:52.584451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caef96d-d73a-40ad-bc07-20d1d9352913 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:52.602243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:52.603810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:52.603959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:52.604246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] PATH_INFO: `/v3/users/9b3057cffda44ff0b4fe28aafb33bda9` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:52.605587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b3057cffda44ff0b4fe28aafb33bda9)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:52.615638 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:52.647850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d15d2-9531-49a9-a50f-c1896e8f6b9c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:52.649239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d15d2-9531-49a9-a50f-c1896e8f6b9c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:52.649671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d15d2-9531-49a9-a50f-c1896e8f6b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:52.657515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:52.682099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-033d15d2-9531-49a9-a50f-c1896e8f6b9c None None] MFA Rules not processed for user `ac1b24e50c894dcf8e887275fd91a174`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:52.683563 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-033d15d2-9531-49a9-a50f-c1896e8f6b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:52.697169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1126/2252] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:52.767780 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b3057cffda44ff0b4fe28aafb33bda9'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:52.786946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b3057cffda44ff0b4fe28aafb33bda9'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:52.787634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b3057cffda44ff0b4fe28aafb33bda9'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:52.833818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b3057cffda44ff0b4fe28aafb33bda9'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:52.850786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b3057cffda44ff0b4fe28aafb33bda9'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:52.862692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1127/2253] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:52 2022] DELETE /identity/v3/users/9b3057cffda44ff0b4fe28aafb33bda9 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:52.867560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31389-eef3-4ab6-97a0-6bef08be05e4 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:52.867847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31389-eef3-4ab6-97a0-6bef08be05e4 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:52.868302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31389-eef3-4ab6-97a0-6bef08be05e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:52.906470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c9a31389-eef3-4ab6-97a0-6bef08be05e4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:52.908770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c9a31389-eef3-4ab6-97a0-6bef08be05e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:53.009672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1127/2254] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:53.020514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:53.022325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:53.129492 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06b68d-eb77-43d4-84b1-1df7d4dbf329 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:53.134192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2698e-bfd5-41a8-a8b2-9fef29834c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:53.135039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2698e-bfd5-41a8-a8b2-9fef29834c3a service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:53.135362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2698e-bfd5-41a8-a8b2-9fef29834c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:53.135831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2698e-bfd5-41a8-a8b2-9fef29834c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:53.147790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2698e-bfd5-41a8-a8b2-9fef29834c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:53.149692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2698e-bfd5-41a8-a8b2-9fef29834c3a service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:53.161420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1128/2255] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:53.606433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a31389-eef3-4ab6-97a0-6bef08be05e4 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:53.609568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c9a31389-eef3-4ab6-97a0-6bef08be05e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:53.644357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a31389-eef3-4ab6-97a0-6bef08be05e4 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:53.670592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:53.673664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:53.674417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:53.674884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] PATH_INFO: `/v3/projects/e56267d619ec479f9b0b705174031570` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:53.708062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e56267d619ec479f9b0b705174031570)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:53.711424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:53.858425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e56267d619ec479f9b0b705174031570', 'request_id': 'req-d405287b-a6a3-49c1-b6e3-8702a7a02af2'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:53.871424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e56267d619ec479f9b0b705174031570', 'request_id': 'req-d405287b-a6a3-49c1-b6e3-8702a7a02af2'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:53.881220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e56267d619ec479f9b0b705174031570', 'request_id': 'req-d405287b-a6a3-49c1-b6e3-8702a7a02af2'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:53.904678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] The token cache is being invalidate because project e56267d619ec479f9b0b705174031570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:53.905035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:53.905962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:53.907761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1128/2256] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:53 2022] DELETE /identity/v3/projects/e56267d619ec479f9b0b705174031570 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:53.914193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2698e-bfd5-41a8-a8b2-9fef29834c3a service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:53.934153 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2698e-bfd5-41a8-a8b2-9fef29834c3a service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:53.947171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:54.073246 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:54.077435 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:54.086680 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fd10d-6ca3-4a5a-9d37-758491ed1546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:54.087945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fd10d-6ca3-4a5a-9d37-758491ed1546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:54.087945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fd10d-6ca3-4a5a-9d37-758491ed1546 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:54.088249 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fd10d-6ca3-4a5a-9d37-758491ed1546 admin admin] PATH_INFO: `/v3/users/97acbfe7996c4e069e7424e52d91e11c` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:54.090304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fd10d-6ca3-4a5a-9d37-758491ed1546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97acbfe7996c4e069e7424e52d91e11c)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:54.093392 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fd10d-6ca3-4a5a-9d37-758491ed1546 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:54.125734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405287b-a6a3-49c1-b6e3-8702a7a02af2 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:54.136691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:54.141741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:54.143280 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:54.143845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] PATH_INFO: `/v3/users/70cdedad5b9f420693ecb1845e380c6a` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:54.155165 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70cdedad5b9f420693ecb1845e380c6a)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:54.170989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:54.203654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ad2fd10d-6ca3-4a5a-9d37-758491ed1546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97acbfe7996c4e069e7424e52d91e11c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:54.218286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ad2fd10d-6ca3-4a5a-9d37-758491ed1546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97acbfe7996c4e069e7424e52d91e11c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:54.219111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ad2fd10d-6ca3-4a5a-9d37-758491ed1546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97acbfe7996c4e069e7424e52d91e11c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:54.242755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ad2fd10d-6ca3-4a5a-9d37-758491ed1546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97acbfe7996c4e069e7424e52d91e11c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:54.264267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ad2fd10d-6ca3-4a5a-9d37-758491ed1546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97acbfe7996c4e069e7424e52d91e11c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:54.291248 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1129/2257] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:53 2022] DELETE /identity/v3/users/97acbfe7996c4e069e7424e52d91e11c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:54.301659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70cdedad5b9f420693ecb1845e380c6a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:54.315028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70cdedad5b9f420693ecb1845e380c6a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:54.346325 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70cdedad5b9f420693ecb1845e380c6a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:54.348778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70cdedad5b9f420693ecb1845e380c6a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:54.381122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:54.395028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70cdedad5b9f420693ecb1845e380c6a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:54.421369 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1129/2258] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:56:54 2022] DELETE /identity/v3/users/70cdedad5b9f420693ecb1845e380c6a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:54.426981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1cb97-cfea-4312-871b-527e5cf9dee7 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:54.427427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1cb97-cfea-4312-871b-527e5cf9dee7 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:54.427768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1cb97-cfea-4312-871b-527e5cf9dee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:54.455870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-61b1cb97-cfea-4312-871b-527e5cf9dee7 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:54.457715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-61b1cb97-cfea-4312-871b-527e5cf9dee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:54.545050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1130/2259] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:54.559621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:54.562130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:54.595098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1cb97-cfea-4312-871b-527e5cf9dee7 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:54.597500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-61b1cb97-cfea-4312-871b-527e5cf9dee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:54.655354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:54.666356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163895c-f87e-45fa-bb29-36a5f37253d8 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:54.672014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806fdc6-35cd-4ac9-afa5-df8eb3e658fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:54.673394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806fdc6-35cd-4ac9-afa5-df8eb3e658fc service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:54.673935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806fdc6-35cd-4ac9-afa5-df8eb3e658fc service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:54.674524 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806fdc6-35cd-4ac9-afa5-df8eb3e658fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:54.689281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806fdc6-35cd-4ac9-afa5-df8eb3e658fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:54.691771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806fdc6-35cd-4ac9-afa5-df8eb3e658fc service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:54.705483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1130/2260] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:54.717107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1cb97-cfea-4312-871b-527e5cf9dee7 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:54.727925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd201391-504a-4e3e-9642-5af70ab18c32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:54.729053 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd201391-504a-4e3e-9642-5af70ab18c32 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:54.730016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd201391-504a-4e3e-9642-5af70ab18c32 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:54.730494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd201391-504a-4e3e-9642-5af70ab18c32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:54.736603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-cd201391-504a-4e3e-9642-5af70ab18c32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:54.890802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd201391-504a-4e3e-9642-5af70ab18c32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:54.894007 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd201391-504a-4e3e-9642-5af70ab18c32 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:54.921834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1131/2261] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:55.132807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806fdc6-35cd-4ac9-afa5-df8eb3e658fc service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:55.134588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1806fdc6-35cd-4ac9-afa5-df8eb3e658fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:55.162520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806fdc6-35cd-4ac9-afa5-df8eb3e658fc service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:55.169113 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:55.170302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:55.170594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:55.170876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] PATH_INFO: `/v3/projects/1566bbe526734c33813b5cdca65fbcfe` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:55.172237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1566bbe526734c33813b5cdca65fbcfe)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:55.174333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:55.268319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1566bbe526734c33813b5cdca65fbcfe', 'request_id': 'req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:55.284579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1566bbe526734c33813b5cdca65fbcfe', 'request_id': 'req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:55.304471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1566bbe526734c33813b5cdca65fbcfe', 'request_id': 'req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:55.344407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] The token cache is being invalidate because project 1566bbe526734c33813b5cdca65fbcfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:55.345383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:55.347733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-de96e93a-f8b0-42c9-9fe9-95e71d044ef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:55.353506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1131/2262] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:55 2022] DELETE /identity/v3/projects/1566bbe526734c33813b5cdca65fbcfe => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:55.358927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5dc61-86c7-496a-808b-356f98d55e2d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:55.359271 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5dc61-86c7-496a-808b-356f98d55e2d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:55.359640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5dc61-86c7-496a-808b-356f98d55e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:55.384829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-fde5dc61-86c7-496a-808b-356f98d55e2d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:55.386108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fde5dc61-86c7-496a-808b-356f98d55e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:55.468191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1132/2263] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:55.478141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4763e-16b7-47f9-adca-3a9c3d4a753c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:55.478532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4763e-16b7-47f9-adca-3a9c3d4a753c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:55.478700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4763e-16b7-47f9-adca-3a9c3d4a753c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:55.519434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-6ca4763e-16b7-47f9-adca-3a9c3d4a753c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:55.520802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6ca4763e-16b7-47f9-adca-3a9c3d4a753c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:55.532469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1132/2264] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:55.536699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5dc61-86c7-496a-808b-356f98d55e2d None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:55.567893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5dc61-86c7-496a-808b-356f98d55e2d None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:55.582385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b339cf1-08a2-4527-b8c9-a4a61536511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:55.583393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b339cf1-08a2-4527-b8c9-a4a61536511a admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:55.583960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b339cf1-08a2-4527-b8c9-a4a61536511a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:55.584535 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b339cf1-08a2-4527-b8c9-a4a61536511a admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:55.586708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b339cf1-08a2-4527-b8c9-a4a61536511a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:55.587452 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b339cf1-08a2-4527-b8c9-a4a61536511a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:55.591171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b339cf1-08a2-4527-b8c9-a4a61536511a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:55.633416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1133/2265] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:56:55.634056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4763e-16b7-47f9-adca-3a9c3d4a753c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:55.645074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4763e-16b7-47f9-adca-3a9c3d4a753c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:55.651160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36f187-1806-4597-8e23-b377ba64c2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:55.652216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36f187-1806-4597-8e23-b377ba64c2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:55.653245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36f187-1806-4597-8e23-b377ba64c2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:55.653245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36f187-1806-4597-8e23-b377ba64c2e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:55.653653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36f187-1806-4597-8e23-b377ba64c2e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:55.655667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36f187-1806-4597-8e23-b377ba64c2e5 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:55.728362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1133/2266] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:55 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:55.736838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b339cf1-08a2-4527-b8c9-a4a61536511a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:55.791889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b339cf1-08a2-4527-b8c9-a4a61536511a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:55.950033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783264dc-cfcd-484d-9e8a-79717b30decd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:55.964569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783264dc-cfcd-484d-9e8a-79717b30decd admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:55.964866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783264dc-cfcd-484d-9e8a-79717b30decd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:55.965324 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783264dc-cfcd-484d-9e8a-79717b30decd admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:55.966261 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783264dc-cfcd-484d-9e8a-79717b30decd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:55.969272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783264dc-cfcd-484d-9e8a-79717b30decd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:55.997250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36f187-1806-4597-8e23-b377ba64c2e5 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:55.999576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1d36f187-1806-4597-8e23-b377ba64c2e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:56.017508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:56.024224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36f187-1806-4597-8e23-b377ba64c2e5 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:56.039700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1134/2267] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:55 2022] POST /identity/v3/users => generated 288 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:56.043812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783264dc-cfcd-484d-9e8a-79717b30decd admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:56.057177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:56.058380 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:56.058823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:56.061119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] PATH_INFO: `/v3/projects/f1930fe918b845209bc20c39f61de60a` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:56.068486 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783264dc-cfcd-484d-9e8a-79717b30decd admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:56.073608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1930fe918b845209bc20c39f61de60a)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:56.076141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:56.094882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c83986-84d0-42dd-aac6-54500d414e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:56.095615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c83986-84d0-42dd-aac6-54500d414e37 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:56.095905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c83986-84d0-42dd-aac6-54500d414e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:56.096192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c83986-84d0-42dd-aac6-54500d414e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:56.098459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c83986-84d0-42dd-aac6-54500d414e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:56.102989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c83986-84d0-42dd-aac6-54500d414e37 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:56.130304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1135/2268] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:56.134823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c83986-84d0-42dd-aac6-54500d414e37 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:56.156902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c83986-84d0-42dd-aac6-54500d414e37 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:56.173929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ff2a9-0e3e-4a65-8aa1-290b636dd998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:56.174619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ff2a9-0e3e-4a65-8aa1-290b636dd998 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:56.175053 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ff2a9-0e3e-4a65-8aa1-290b636dd998 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:56.175341 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ff2a9-0e3e-4a65-8aa1-290b636dd998 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:56.176165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ff2a9-0e3e-4a65-8aa1-290b636dd998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:56.177997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ff2a9-0e3e-4a65-8aa1-290b636dd998 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:56.207571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1136/2269] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:56:56.217351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ff2a9-0e3e-4a65-8aa1-290b636dd998 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:56.256484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ff2a9-0e3e-4a65-8aa1-290b636dd998 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:56.259184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:56.265175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1930fe918b845209bc20c39f61de60a', 'request_id': 'req-6660073a-adc7-4ac9-9a41-88575e286e77'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:56.288783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1930fe918b845209bc20c39f61de60a', 'request_id': 'req-6660073a-adc7-4ac9-9a41-88575e286e77'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:56.302036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1930fe918b845209bc20c39f61de60a', 'request_id': 'req-6660073a-adc7-4ac9-9a41-88575e286e77'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:56.333429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] The token cache is being invalidate because project f1930fe918b845209bc20c39f61de60a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:56:56.333883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:56.335362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6660073a-adc7-4ac9-9a41-88575e286e77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:56:56.339462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1134/2270] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:56:55 2022] DELETE /identity/v3/projects/f1930fe918b845209bc20c39f61de60a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:56.346274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261a5fb-7e3a-4d1c-b47f-284f9155a418 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:56.348286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261a5fb-7e3a-4d1c-b47f-284f9155a418 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:56.348970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261a5fb-7e3a-4d1c-b47f-284f9155a418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:56.410498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-2261a5fb-7e3a-4d1c-b47f-284f9155a418 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:56.412312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2261a5fb-7e3a-4d1c-b47f-284f9155a418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:56.442191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbda75-72a2-4867-bc70-ced362cea664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:56.445095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbda75-72a2-4867-bc70-ced362cea664 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:56.445525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbda75-72a2-4867-bc70-ced362cea664 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:56.446023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbda75-72a2-4867-bc70-ced362cea664 admin admin] PATH_INFO: `/v3/projects/1a8b1e9a00a84d008d995b41221e7b52/users/db8aeee666254299877e51f90ff53e71/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:56.482720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbda75-72a2-4867-bc70-ced362cea664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8b1e9a00a84d008d995b41221e7b52, user_id=db8aeee666254299877e51f90ff53e71, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:56.485763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbda75-72a2-4867-bc70-ced362cea664 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:56.528385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1137/2271] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:56 2022] PUT /identity/v3/projects/1a8b1e9a00a84d008d995b41221e7b52/users/db8aeee666254299877e51f90ff53e71/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:56:56.535650 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050dd34-e080-4395-9e98-fa6cb58242fd None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:56.536981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050dd34-e080-4395-9e98-fa6cb58242fd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:56.537272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050dd34-e080-4395-9e98-fa6cb58242fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:56.576851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e050dd34-e080-4395-9e98-fa6cb58242fd None None] MFA Rules not processed for user `db8aeee666254299877e51f90ff53e71`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:56.578090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e050dd34-e080-4395-9e98-fa6cb58242fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:56.632601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1135/2272] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:56.650558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbdc87-6734-4017-b5e1-97528bab119c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:56.651020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbdc87-6734-4017-b5e1-97528bab119c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:56.651404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbdc87-6734-4017-b5e1-97528bab119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:56.680163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1138/2273] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:56:56.735136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c8bbdc87-6734-4017-b5e1-97528bab119c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:56.737787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c8bbdc87-6734-4017-b5e1-97528bab119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:56.839609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050dd34-e080-4395-9e98-fa6cb58242fd None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:56.844052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e050dd34-e080-4395-9e98-fa6cb58242fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:56.959179 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1136/2274] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:56.964247 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbdc87-6734-4017-b5e1-97528bab119c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:56.986833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbdc87-6734-4017-b5e1-97528bab119c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:57.003628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6227b8-b04b-48ec-b8d5-8b92b40a3c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:57.005423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6227b8-b04b-48ec-b8d5-8b92b40a3c66 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:57.006034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6227b8-b04b-48ec-b8d5-8b92b40a3c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:57.006506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6227b8-b04b-48ec-b8d5-8b92b40a3c66 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:57.007518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6227b8-b04b-48ec-b8d5-8b92b40a3c66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:56:57.010310 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6227b8-b04b-48ec-b8d5-8b92b40a3c66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:57.014445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6227b8-b04b-48ec-b8d5-8b92b40a3c66 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:57.030469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:57.048581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050dd34-e080-4395-9e98-fa6cb58242fd None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:57.049524 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1137/2275] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:56:57.054146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6227b8-b04b-48ec-b8d5-8b92b40a3c66 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:57.066804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc042e2-86d3-4e70-9fc0-e5d61cded2ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:57.067891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc042e2-86d3-4e70-9fc0-e5d61cded2ce service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:57.068426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc042e2-86d3-4e70-9fc0-e5d61cded2ce service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:57.068916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc042e2-86d3-4e70-9fc0-e5d61cded2ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:57.087721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6227b8-b04b-48ec-b8d5-8b92b40a3c66 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:57.092268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc042e2-86d3-4e70-9fc0-e5d61cded2ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:57.096453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc042e2-86d3-4e70-9fc0-e5d61cded2ce service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:57.101401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af33fa-33c7-434e-bf18-2d0ab3fa64ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:57.103623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af33fa-33c7-434e-bf18-2d0ab3fa64ca admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:57.106684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af33fa-33c7-434e-bf18-2d0ab3fa64ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:57.107763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af33fa-33c7-434e-bf18-2d0ab3fa64ca admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:57.109168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af33fa-33c7-434e-bf18-2d0ab3fa64ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:57.113026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af33fa-33c7-434e-bf18-2d0ab3fa64ca admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:57.124143 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1139/2276] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:56:57.157990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1138/2277] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:56:57 2022] POST /identity/v3/projects => generated 391 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:57.162297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc042e2-86d3-4e70-9fc0-e5d61cded2ce service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:57.180768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc042e2-86d3-4e70-9fc0-e5d61cded2ce service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:57.514040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdf345-ef6b-4775-b7e4-f14471721c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:57.514856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdf345-ef6b-4775-b7e4-f14471721c35 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:57.515361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdf345-ef6b-4775-b7e4-f14471721c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:57.515911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdf345-ef6b-4775-b7e4-f14471721c35 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:57.527077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdf345-ef6b-4775-b7e4-f14471721c35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:57.544454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdf345-ef6b-4775-b7e4-f14471721c35 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:57.655317 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af33fa-33c7-434e-bf18-2d0ab3fa64ca admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:57.659447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1140/2278] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:56:57 2022] POST /identity/v3/users => generated 296 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:56:57.694620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af33fa-33c7-434e-bf18-2d0ab3fa64ca admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:57.736654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dc545-6c66-404c-b27d-7e396ab77955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:57.744699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dc545-6c66-404c-b27d-7e396ab77955 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:57.745014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dc545-6c66-404c-b27d-7e396ab77955 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:57.751082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dc545-6c66-404c-b27d-7e396ab77955 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:57.753945 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dc545-6c66-404c-b27d-7e396ab77955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:57.768429 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dc545-6c66-404c-b27d-7e396ab77955 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:57.821582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1139/2279] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:57.826736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdf345-ef6b-4775-b7e4-f14471721c35 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:57.851812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdf345-ef6b-4775-b7e4-f14471721c35 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:57.876302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380efcf-1252-4baf-9864-e084df0fe271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:57.879348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380efcf-1252-4baf-9864-e084df0fe271 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:57.879673 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380efcf-1252-4baf-9864-e084df0fe271 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:57.880528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380efcf-1252-4baf-9864-e084df0fe271 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:57.882017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380efcf-1252-4baf-9864-e084df0fe271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:57.887439 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380efcf-1252-4baf-9864-e084df0fe271 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:57.945626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dc545-6c66-404c-b27d-7e396ab77955 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:57.949443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1141/2280] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:56:57.972048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dc545-6c66-404c-b27d-7e396ab77955 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:57.992289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353cf50-3e5b-4054-9493-8122657a57d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:57.993545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353cf50-3e5b-4054-9493-8122657a57d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:57.994082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353cf50-3e5b-4054-9493-8122657a57d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:57.994662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353cf50-3e5b-4054-9493-8122657a57d7 admin admin] PATH_INFO: `/v3/projects/4faf93fcafba4617b380538f6a9a9305/users/52661b146433497c9d6c03bc949a5ada/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:58.068021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353cf50-3e5b-4054-9493-8122657a57d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4faf93fcafba4617b380538f6a9a9305, user_id=52661b146433497c9d6c03bc949a5ada, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:58.083755 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353cf50-3e5b-4054-9493-8122657a57d7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:58.135242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1140/2281] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:56:57 2022] PUT /identity/v3/projects/4faf93fcafba4617b380538f6a9a9305/users/52661b146433497c9d6c03bc949a5ada/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:56:58.161454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb7896-d896-4069-a0d5-0877d3c7177e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:58.162067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb7896-d896-4069-a0d5-0877d3c7177e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:58.162238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb7896-d896-4069-a0d5-0877d3c7177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:58.216047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-cfeb7896-d896-4069-a0d5-0877d3c7177e None None] MFA Rules not processed for user `52661b146433497c9d6c03bc949a5ada`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:56:58.218196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-cfeb7896-d896-4069-a0d5-0877d3c7177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:58.457270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1142/2282] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:56:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:56:58.473156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353cf50-3e5b-4054-9493-8122657a57d7 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:56:58.475169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b353cf50-3e5b-4054-9493-8122657a57d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:56:58.484455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:56:58.634608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353cf50-3e5b-4054-9493-8122657a57d7 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:56:58.639874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacad64a-ad07-4d97-bdb2-ecad61ea0ea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:56:58.645380 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacad64a-ad07-4d97-bdb2-ecad61ea0ea6 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:56:58.645829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacad64a-ad07-4d97-bdb2-ecad61ea0ea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:56:58.646123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacad64a-ad07-4d97-bdb2-ecad61ea0ea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:56:58.657503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacad64a-ad07-4d97-bdb2-ecad61ea0ea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:56:58.659510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacad64a-ad07-4d97-bdb2-ecad61ea0ea6 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:56:58.681571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1141/2283] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:56:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:57:02.032220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb7896-d896-4069-a0d5-0877d3c7177e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:02.033855 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-cfeb7896-d896-4069-a0d5-0877d3c7177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:02.117523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:02.123628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb7896-d896-4069-a0d5-0877d3c7177e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:02.132443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:02.133401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:02.135797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:02.136135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] PATH_INFO: `/v3/users/db8aeee666254299877e51f90ff53e71` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:02.137555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8aeee666254299877e51f90ff53e71)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:02.139599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:02.192646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8aeee666254299877e51f90ff53e71'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:02.200204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8aeee666254299877e51f90ff53e71'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:02.201038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8aeee666254299877e51f90ff53e71'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:02.218364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8aeee666254299877e51f90ff53e71'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:02.225402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8aeee666254299877e51f90ff53e71'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:02.232640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1143/2284] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:57:02 2022] DELETE /identity/v3/users/db8aeee666254299877e51f90ff53e71 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:02.237303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9dcbe-cf6b-4bcb-88ee-eca783c87ee4 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:02.237934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9dcbe-cf6b-4bcb-88ee-eca783c87ee4 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:02.238043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9dcbe-cf6b-4bcb-88ee-eca783c87ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:02.256684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-84e9dcbe-cf6b-4bcb-88ee-eca783c87ee4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:02.257904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-84e9dcbe-cf6b-4bcb-88ee-eca783c87ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:02.322206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1142/2285] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:02.332391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:02.333752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:02.395208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22fd15-5802-4ab9-b420-e95e38991725 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:02.398885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1a4ae-5a98-407f-911f-03507201ae43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:02.399540 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1a4ae-5a98-407f-911f-03507201ae43 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:02.399821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1a4ae-5a98-407f-911f-03507201ae43 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:02.400092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1a4ae-5a98-407f-911f-03507201ae43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:02.410496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1a4ae-5a98-407f-911f-03507201ae43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:02.412135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1a4ae-5a98-407f-911f-03507201ae43 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:02.423593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1144/2286] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:57:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:57:02.723687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9dcbe-cf6b-4bcb-88ee-eca783c87ee4 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:02.725707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-84e9dcbe-cf6b-4bcb-88ee-eca783c87ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:02.739361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9dcbe-cf6b-4bcb-88ee-eca783c87ee4 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:02.744782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:02.746006 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:02.746293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:02.746570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] PATH_INFO: `/v3/projects/1a8b1e9a00a84d008d995b41221e7b52` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:02.747842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a8b1e9a00a84d008d995b41221e7b52)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:02.749856 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:02.801773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a8b1e9a00a84d008d995b41221e7b52', 'request_id': 'req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:02.809488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a8b1e9a00a84d008d995b41221e7b52', 'request_id': 'req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:02.816534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a8b1e9a00a84d008d995b41221e7b52', 'request_id': 'req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:02.828393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] The token cache is being invalidate because project 1a8b1e9a00a84d008d995b41221e7b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:57:02.828835 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:02.829238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-08d2c5ef-4a29-4ab9-8573-9218bdd7b460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:02.830820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1143/2287] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:57:02 2022] DELETE /identity/v3/projects/1a8b1e9a00a84d008d995b41221e7b52 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:02.836134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c6e1b-3cf9-465a-a999-e3dea15fb6f8 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:02.836499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c6e1b-3cf9-465a-a999-e3dea15fb6f8 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:02.836789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c6e1b-3cf9-465a-a999-e3dea15fb6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:02.860985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-4d5c6e1b-3cf9-465a-a999-e3dea15fb6f8 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:02.862282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4d5c6e1b-3cf9-465a-a999-e3dea15fb6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:02.932743 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1145/2288] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:02.941080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db42cec-58a6-4a49-85d3-ac74f5ad5c07 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:02.941286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db42cec-58a6-4a49-85d3-ac74f5ad5c07 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:02.941521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db42cec-58a6-4a49-85d3-ac74f5ad5c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:02.961346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3db42cec-58a6-4a49-85d3-ac74f5ad5c07 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:02.962726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3db42cec-58a6-4a49-85d3-ac74f5ad5c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:02.976062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1144/2289] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:02.980763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c6e1b-3cf9-465a-a999-e3dea15fb6f8 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:02.990787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c6e1b-3cf9-465a-a999-e3dea15fb6f8 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:02.995449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba4d6b-5fbd-4351-8c94-63ae51d1485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:02.996115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba4d6b-5fbd-4351-8c94-63ae51d1485d admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:02.996453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba4d6b-5fbd-4351-8c94-63ae51d1485d admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:02.996730 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba4d6b-5fbd-4351-8c94-63ae51d1485d admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:02.997453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba4d6b-5fbd-4351-8c94-63ae51d1485d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:57:02.997760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba4d6b-5fbd-4351-8c94-63ae51d1485d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:02.999344 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba4d6b-5fbd-4351-8c94-63ae51d1485d admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:03.015401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1146/2290] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:57:03.017070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db42cec-58a6-4a49-85d3-ac74f5ad5c07 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:03.027285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db42cec-58a6-4a49-85d3-ac74f5ad5c07 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:03.034264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66041d0-803e-4e54-b14d-c3072da85eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:03.034959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66041d0-803e-4e54-b14d-c3072da85eef admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:03.035239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66041d0-803e-4e54-b14d-c3072da85eef admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:03.035514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66041d0-803e-4e54-b14d-c3072da85eef admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:03.036249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66041d0-803e-4e54-b14d-c3072da85eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:03.038184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66041d0-803e-4e54-b14d-c3072da85eef admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:03.058154 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:03.068354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1145/2291] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:57:03 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:57:03.071561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba4d6b-5fbd-4351-8c94-63ae51d1485d admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:03.082788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba4d6b-5fbd-4351-8c94-63ae51d1485d admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:03.086699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:03.144922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bbcc7-3213-4909-aeb1-f92b758cab53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:03.145606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bbcc7-3213-4909-aeb1-f92b758cab53 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:03.145893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bbcc7-3213-4909-aeb1-f92b758cab53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:03.146178 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bbcc7-3213-4909-aeb1-f92b758cab53 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:03.146876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bbcc7-3213-4909-aeb1-f92b758cab53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:03.148831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bbcc7-3213-4909-aeb1-f92b758cab53 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:03.175694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1147/2292] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:57:03 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:57:03.186406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66041d0-803e-4e54-b14d-c3072da85eef admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:03.199047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66041d0-803e-4e54-b14d-c3072da85eef admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:03.207473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe979ba1-15a8-47bd-bde6-222f8f09de0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:03.209135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe979ba1-15a8-47bd-bde6-222f8f09de0e admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:03.209618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe979ba1-15a8-47bd-bde6-222f8f09de0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:03.210144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe979ba1-15a8-47bd-bde6-222f8f09de0e admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:03.212839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe979ba1-15a8-47bd-bde6-222f8f09de0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:03.216092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe979ba1-15a8-47bd-bde6-222f8f09de0e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:03.230428 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1146/2293] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:57:03.235472 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bbcc7-3213-4909-aeb1-f92b758cab53 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:03.246184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bbcc7-3213-4909-aeb1-f92b758cab53 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:03.254404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c4241-1455-4e16-be82-c824cd1104d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:03.255761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4241-1455-4e16-be82-c824cd1104d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:03.256203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4241-1455-4e16-be82-c824cd1104d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:03.256856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4241-1455-4e16-be82-c824cd1104d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:03.257613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c4241-1455-4e16-be82-c824cd1104d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:03.259597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c4241-1455-4e16-be82-c824cd1104d3 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:03.270939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1148/2294] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:57:03.275386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe979ba1-15a8-47bd-bde6-222f8f09de0e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:03.288005 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe979ba1-15a8-47bd-bde6-222f8f09de0e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:03.295545 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e78e0-c47d-4ef2-b709-07b7251c1345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:03.296471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e78e0-c47d-4ef2-b709-07b7251c1345 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:03.296753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e78e0-c47d-4ef2-b709-07b7251c1345 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:03.297036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e78e0-c47d-4ef2-b709-07b7251c1345 admin admin] PATH_INFO: `/v3/projects/4ad611122b054a2eb44fc11b323b6c5f/users/cd4f05aa641f47f5b1f5ff41ca984a10/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:03.313048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e78e0-c47d-4ef2-b709-07b7251c1345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad611122b054a2eb44fc11b323b6c5f, user_id=cd4f05aa641f47f5b1f5ff41ca984a10, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:03.315978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e78e0-c47d-4ef2-b709-07b7251c1345 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:03.330972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1147/2295] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:57:03 2022] PUT /identity/v3/projects/4ad611122b054a2eb44fc11b323b6c5f/users/cd4f05aa641f47f5b1f5ff41ca984a10/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:03.400595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fa7ee-476a-4e6d-988e-f7e5e0738803 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:03.400889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fa7ee-476a-4e6d-988e-f7e5e0738803 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:03.401173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fa7ee-476a-4e6d-988e-f7e5e0738803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:03.432158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-9b9fa7ee-476a-4e6d-988e-f7e5e0738803 None None] MFA Rules not processed for user `cd4f05aa641f47f5b1f5ff41ca984a10`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:03.433430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9b9fa7ee-476a-4e6d-988e-f7e5e0738803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:03.501995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1149/2296] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:03.513636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e78e0-c47d-4ef2-b709-07b7251c1345 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:03.515293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-234e78e0-c47d-4ef2-b709-07b7251c1345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:03.578381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e78e0-c47d-4ef2-b709-07b7251c1345 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:03.582263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85421e3-6c59-4021-8e60-1aef833a7f1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:03.582963 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85421e3-6c59-4021-8e60-1aef833a7f1c service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:03.583226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85421e3-6c59-4021-8e60-1aef833a7f1c service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:03.583510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85421e3-6c59-4021-8e60-1aef833a7f1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:03.594668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85421e3-6c59-4021-8e60-1aef833a7f1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:03.596156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85421e3-6c59-4021-8e60-1aef833a7f1c service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:03.607093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1148/2297] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:57:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:57:13.498903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:13.602181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:15.688320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fa7ee-476a-4e6d-988e-f7e5e0738803 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:15.690072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9b9fa7ee-476a-4e6d-988e-f7e5e0738803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:15.768199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fa7ee-476a-4e6d-988e-f7e5e0738803 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:15.774798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:15.775703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:15.775992 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:15.776276 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] PATH_INFO: `/v3/users/34a5a1913029485c9b4b38a8b4b8e149` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:15.777542 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a5a1913029485c9b4b38a8b4b8e149)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:15.780182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:15.838448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a5a1913029485c9b4b38a8b4b8e149'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:15.849004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a5a1913029485c9b4b38a8b4b8e149'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:15.849659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a5a1913029485c9b4b38a8b4b8e149'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:15.864834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a5a1913029485c9b4b38a8b4b8e149'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:15.877406 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a5a1913029485c9b4b38a8b4b8e149'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:15.884853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1150/2298] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:57:15 2022] DELETE /identity/v3/users/34a5a1913029485c9b4b38a8b4b8e149 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:15.890185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4191ee7-db62-4c15-b5a8-44726ad3f350 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:15.890683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4191ee7-db62-4c15-b5a8-44726ad3f350 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:15.890778 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4191ee7-db62-4c15-b5a8-44726ad3f350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:15.912175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-b4191ee7-db62-4c15-b5a8-44726ad3f350 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:15.913451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b4191ee7-db62-4c15-b5a8-44726ad3f350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:15.997289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1149/2299] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:57:16.007243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:16.008835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:16.081260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36d6d3-8b8e-4925-967b-fd208b1a04b4 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:16.085315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00e76e-6bf6-4d36-80ae-87c9a7e86aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:16.085974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00e76e-6bf6-4d36-80ae-87c9a7e86aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:16.086258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00e76e-6bf6-4d36-80ae-87c9a7e86aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:16.086535 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00e76e-6bf6-4d36-80ae-87c9a7e86aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:16.098653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00e76e-6bf6-4d36-80ae-87c9a7e86aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:16.100022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00e76e-6bf6-4d36-80ae-87c9a7e86aa0 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:16.111633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1151/2300] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:57:16.505597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4191ee7-db62-4c15-b5a8-44726ad3f350 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:16.507577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b4191ee7-db62-4c15-b5a8-44726ad3f350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:16.521699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4191ee7-db62-4c15-b5a8-44726ad3f350 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:16.528346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:16.529732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:16.530237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:16.530469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] PATH_INFO: `/v3/projects/b3e57460f4dd42ce812e3f304d1932ed` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:16.531747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3e57460f4dd42ce812e3f304d1932ed)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:16.533971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:16.584476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3e57460f4dd42ce812e3f304d1932ed', 'request_id': 'req-19590228-a219-41fb-a99f-d018e3f1c4f5'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:16.592469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3e57460f4dd42ce812e3f304d1932ed', 'request_id': 'req-19590228-a219-41fb-a99f-d018e3f1c4f5'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:16.599349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3e57460f4dd42ce812e3f304d1932ed', 'request_id': 'req-19590228-a219-41fb-a99f-d018e3f1c4f5'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:16.610448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] The token cache is being invalidate because project b3e57460f4dd42ce812e3f304d1932ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:57:16.610743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:16.611440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:16.613086 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1150/2301] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:57:16 2022] DELETE /identity/v3/projects/b3e57460f4dd42ce812e3f304d1932ed => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:16.617104 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00e76e-6bf6-4d36-80ae-87c9a7e86aa0 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:16.627367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00e76e-6bf6-4d36-80ae-87c9a7e86aa0 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:16.646345 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:16.689015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88933875-e273-4fe6-8b07-830b5ba57151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:16.690159 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933875-e273-4fe6-8b07-830b5ba57151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:16.690442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933875-e273-4fe6-8b07-830b5ba57151 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:16.690718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933875-e273-4fe6-8b07-830b5ba57151 admin admin] PATH_INFO: `/v3/users/9b1b157194524c59b092dc390c589c93` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:16.691935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88933875-e273-4fe6-8b07-830b5ba57151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b1b157194524c59b092dc390c589c93)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:16.693905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88933875-e273-4fe6-8b07-830b5ba57151 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:16.743742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-88933875-e273-4fe6-8b07-830b5ba57151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b1b157194524c59b092dc390c589c93'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:16.752093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-88933875-e273-4fe6-8b07-830b5ba57151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b1b157194524c59b092dc390c589c93'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:16.752871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-88933875-e273-4fe6-8b07-830b5ba57151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b1b157194524c59b092dc390c589c93'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:16.766202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-88933875-e273-4fe6-8b07-830b5ba57151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b1b157194524c59b092dc390c589c93'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:16.774753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-88933875-e273-4fe6-8b07-830b5ba57151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b1b157194524c59b092dc390c589c93'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:16.781101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1152/2302] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:57:16 2022] DELETE /identity/v3/users/9b1b157194524c59b092dc390c589c93 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:17.063119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:17.065057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:17.082068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19590228-a219-41fb-a99f-d018e3f1c4f5 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:17.141385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:17.142253 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:17.142531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:17.142807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] PATH_INFO: `/v3/projects/33cfa18230914e2b90e860ed75da8f0c` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:17.144025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33cfa18230914e2b90e860ed75da8f0c)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:17.146000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:17.178037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:17.197499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33cfa18230914e2b90e860ed75da8f0c', 'request_id': 'req-69ad6257-5bcd-40df-9239-8fd55875d463'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:17.205471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33cfa18230914e2b90e860ed75da8f0c', 'request_id': 'req-69ad6257-5bcd-40df-9239-8fd55875d463'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:17.213350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33cfa18230914e2b90e860ed75da8f0c', 'request_id': 'req-69ad6257-5bcd-40df-9239-8fd55875d463'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:17.224665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] The token cache is being invalidate because project 33cfa18230914e2b90e860ed75da8f0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:57:17.225037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:17.225989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-69ad6257-5bcd-40df-9239-8fd55875d463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:17.227618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1151/2303] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:57:17 2022] DELETE /identity/v3/projects/33cfa18230914e2b90e860ed75da8f0c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:17.234304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0602774-eb36-423a-81cc-a5a11b0e6b29 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:17.234595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0602774-eb36-423a-81cc-a5a11b0e6b29 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:17.234871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0602774-eb36-423a-81cc-a5a11b0e6b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:17.266797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-b0602774-eb36-423a-81cc-a5a11b0e6b29 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:17.268026 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b0602774-eb36-423a-81cc-a5a11b0e6b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:17.337234 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1153/2304] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:57:17.347309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3712b9c-5c08-4116-8aa1-dfda86822e00 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:17.347603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3712b9c-5c08-4116-8aa1-dfda86822e00 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:17.347886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3712b9c-5c08-4116-8aa1-dfda86822e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:17.373236 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-a3712b9c-5c08-4116-8aa1-dfda86822e00 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:17.374463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a3712b9c-5c08-4116-8aa1-dfda86822e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:17.382691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1152/2305] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:57:17.387134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0602774-eb36-423a-81cc-a5a11b0e6b29 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:17.397559 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0602774-eb36-423a-81cc-a5a11b0e6b29 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:17.402301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f765d-2d28-49bf-9fd0-fa8f07804e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:17.402991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f765d-2d28-49bf-9fd0-fa8f07804e45 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:17.403281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f765d-2d28-49bf-9fd0-fa8f07804e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:17.404084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f765d-2d28-49bf-9fd0-fa8f07804e45 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:17.404858 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f765d-2d28-49bf-9fd0-fa8f07804e45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:57:17.405171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f765d-2d28-49bf-9fd0-fa8f07804e45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:17.406778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f765d-2d28-49bf-9fd0-fa8f07804e45 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:17.418753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1154/2306] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 20:57:17.425852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3712b9c-5c08-4116-8aa1-dfda86822e00 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:17.435665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3712b9c-5c08-4116-8aa1-dfda86822e00 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:17.440975 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e88ae5-416d-40e2-a7a1-84726802c44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:17.441663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e88ae5-416d-40e2-a7a1-84726802c44e admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:17.442064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e88ae5-416d-40e2-a7a1-84726802c44e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:17.442346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e88ae5-416d-40e2-a7a1-84726802c44e admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:17.443041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e88ae5-416d-40e2-a7a1-84726802c44e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:17.444970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e88ae5-416d-40e2-a7a1-84726802c44e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:17.473705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1153/2307] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:57:17 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:57:17.477643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f765d-2d28-49bf-9fd0-fa8f07804e45 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:17.489781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f765d-2d28-49bf-9fd0-fa8f07804e45 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:17.542264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:17.584785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0544173-50a5-4b06-8497-13f055eb43e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:17.585481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0544173-50a5-4b06-8497-13f055eb43e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:17.585770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0544173-50a5-4b06-8497-13f055eb43e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:17.586049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0544173-50a5-4b06-8497-13f055eb43e3 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:17.586768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0544173-50a5-4b06-8497-13f055eb43e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:17.588830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0544173-50a5-4b06-8497-13f055eb43e3 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:17.616715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1155/2308] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:57:17 2022] POST /identity/v3/users => generated 286 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:57:17.617683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e88ae5-416d-40e2-a7a1-84726802c44e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:17.632785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e88ae5-416d-40e2-a7a1-84726802c44e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:17.637601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600592b-7cfb-4f38-b636-fccb37994625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:17.638305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600592b-7cfb-4f38-b636-fccb37994625 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:17.638587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600592b-7cfb-4f38-b636-fccb37994625 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:17.638861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600592b-7cfb-4f38-b636-fccb37994625 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:17.639618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600592b-7cfb-4f38-b636-fccb37994625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:17.641256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600592b-7cfb-4f38-b636-fccb37994625 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:17.651867 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1154/2309] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:57:17.656480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0544173-50a5-4b06-8497-13f055eb43e3 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:17.668014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0544173-50a5-4b06-8497-13f055eb43e3 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:17.673580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcee09b-eb8e-4015-8a0c-4e73fccbfdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:17.674247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcee09b-eb8e-4015-8a0c-4e73fccbfdeb admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:17.674539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcee09b-eb8e-4015-8a0c-4e73fccbfdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:17.675206 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcee09b-eb8e-4015-8a0c-4e73fccbfdeb admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:17.675957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcee09b-eb8e-4015-8a0c-4e73fccbfdeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:17.677603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcee09b-eb8e-4015-8a0c-4e73fccbfdeb admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:17.688725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1156/2310] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:57:17.693390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600592b-7cfb-4f38-b636-fccb37994625 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:17.704772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600592b-7cfb-4f38-b636-fccb37994625 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:17.713637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0d84d-9c0f-4834-88bd-ea9fc8d6915f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:17.714571 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0d84d-9c0f-4834-88bd-ea9fc8d6915f admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:17.714875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0d84d-9c0f-4834-88bd-ea9fc8d6915f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:17.715207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0d84d-9c0f-4834-88bd-ea9fc8d6915f admin admin] PATH_INFO: `/v3/projects/051e2bfc96334054956366e1308d067d/users/72aaaf0bf8ee461c83350055d207951e/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:17.731214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0d84d-9c0f-4834-88bd-ea9fc8d6915f admin admin] RBAC: Authorizing `identity:create_grant(project_id=051e2bfc96334054956366e1308d067d, user_id=72aaaf0bf8ee461c83350055d207951e, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:17.733884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0d84d-9c0f-4834-88bd-ea9fc8d6915f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:17.747415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1155/2311] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:57:17 2022] PUT /identity/v3/projects/051e2bfc96334054956366e1308d067d/users/72aaaf0bf8ee461c83350055d207951e/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:17.752701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b24b1-e4d7-4a46-8a68-d8a7d0920ca1 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:17.753183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b24b1-e4d7-4a46-8a68-d8a7d0920ca1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:17.753309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b24b1-e4d7-4a46-8a68-d8a7d0920ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:17.771707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-796b24b1-e4d7-4a46-8a68-d8a7d0920ca1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:17.772968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-796b24b1-e4d7-4a46-8a68-d8a7d0920ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:17.833894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1157/2312] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:17.846874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0d84d-9c0f-4834-88bd-ea9fc8d6915f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:17.849312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-caf0d84d-9c0f-4834-88bd-ea9fc8d6915f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:17.911091 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0d84d-9c0f-4834-88bd-ea9fc8d6915f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:17.916026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66e297-9307-4b56-835c-54750e340320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:17.916747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66e297-9307-4b56-835c-54750e340320 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:17.917026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66e297-9307-4b56-835c-54750e340320 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:17.917323 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66e297-9307-4b56-835c-54750e340320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:17.927543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66e297-9307-4b56-835c-54750e340320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:17.929360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66e297-9307-4b56-835c-54750e340320 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:17.940676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1156/2313] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:57:18.621003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a5c3c-53cc-47c0-886b-cd3aa3e38515 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:18.621470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a5c3c-53cc-47c0-886b-cd3aa3e38515 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:18.621941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a5c3c-53cc-47c0-886b-cd3aa3e38515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:18.664982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-dd6a5c3c-53cc-47c0-886b-cd3aa3e38515 None None] MFA Rules not processed for user `72aaaf0bf8ee461c83350055d207951e`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:18.666336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-dd6a5c3c-53cc-47c0-886b-cd3aa3e38515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:18.768885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1158/2314] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:18.776048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66e297-9307-4b56-835c-54750e340320 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:18.777774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-5d66e297-9307-4b56-835c-54750e340320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:18.858613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66e297-9307-4b56-835c-54750e340320 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:18.862666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527515e-b46b-412c-af48-417aa1ad824c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:18.863449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527515e-b46b-412c-af48-417aa1ad824c service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:18.863774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527515e-b46b-412c-af48-417aa1ad824c service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:18.864057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527515e-b46b-412c-af48-417aa1ad824c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:18.871206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:18.873798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527515e-b46b-412c-af48-417aa1ad824c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:18.875706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527515e-b46b-412c-af48-417aa1ad824c service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:18.886983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1157/2315] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:57:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:57:23.427957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf85e5-cd19-47f7-baf3-c45211a73b63 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:23.428557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf85e5-cd19-47f7-baf3-c45211a73b63 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:23.428941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf85e5-cd19-47f7-baf3-c45211a73b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:23.464954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-2dbf85e5-cd19-47f7-baf3-c45211a73b63 None None] MFA Rules not processed for user `72aaaf0bf8ee461c83350055d207951e`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:23.466700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2dbf85e5-cd19-47f7-baf3-c45211a73b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:23.492394 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1159/2316] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:25.370356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527515e-b46b-412c-af48-417aa1ad824c service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:25.372764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f527515e-b46b-412c-af48-417aa1ad824c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:25.486279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527515e-b46b-412c-af48-417aa1ad824c service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:25.492179 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd15c2-28bc-4a71-9e7a-357b7c76bf3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:25.492893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd15c2-28bc-4a71-9e7a-357b7c76bf3e service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:25.493231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd15c2-28bc-4a71-9e7a-357b7c76bf3e service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:25.493575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd15c2-28bc-4a71-9e7a-357b7c76bf3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:25.495385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-36fd15c2-28bc-4a71-9e7a-357b7c76bf3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:25.514902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd15c2-28bc-4a71-9e7a-357b7c76bf3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:25.516570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd15c2-28bc-4a71-9e7a-357b7c76bf3e service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:25.531538 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1158/2317] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:57:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:57:27.975541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf85e5-cd19-47f7-baf3-c45211a73b63 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:27.977119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2dbf85e5-cd19-47f7-baf3-c45211a73b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:27.994309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf85e5-cd19-47f7-baf3-c45211a73b63 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:27.999478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:28.000483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:28.000675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:28.000928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] PATH_INFO: `/v3/users/52661b146433497c9d6c03bc949a5ada` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:28.002130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52661b146433497c9d6c03bc949a5ada)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:28.004023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:28.036252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:28.058255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52661b146433497c9d6c03bc949a5ada'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:28.066826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52661b146433497c9d6c03bc949a5ada'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:28.068184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52661b146433497c9d6c03bc949a5ada'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:28.084243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52661b146433497c9d6c03bc949a5ada'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:28.091316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52661b146433497c9d6c03bc949a5ada'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:28.097763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1160/2318] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:57:27 2022] DELETE /identity/v3/users/52661b146433497c9d6c03bc949a5ada => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:28.102397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab855630-a840-4aa1-b5cc-1b70fa852e67 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:28.102875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab855630-a840-4aa1-b5cc-1b70fa852e67 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:28.102993 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab855630-a840-4aa1-b5cc-1b70fa852e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:28.120383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-ab855630-a840-4aa1-b5cc-1b70fa852e67 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:28.121601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ab855630-a840-4aa1-b5cc-1b70fa852e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:28.186676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1159/2319] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:28.196030 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:28.198034 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:28.272692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1b7f3-e65a-4935-9911-72b51c1163b8 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:28.276613 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55457e78-116c-485d-818d-d35d323b6a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:28.277277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457e78-116c-485d-818d-d35d323b6a6d service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:28.277563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457e78-116c-485d-818d-d35d323b6a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:28.277846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457e78-116c-485d-818d-d35d323b6a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:28.287554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55457e78-116c-485d-818d-d35d323b6a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:28.289037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55457e78-116c-485d-818d-d35d323b6a6d service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:28.300303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1161/2320] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:57:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:57:28.653893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab855630-a840-4aa1-b5cc-1b70fa852e67 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:28.656057 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ab855630-a840-4aa1-b5cc-1b70fa852e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:28.668474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab855630-a840-4aa1-b5cc-1b70fa852e67 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:28.673756 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:28.674621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:28.675438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:28.675438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] PATH_INFO: `/v3/projects/4faf93fcafba4617b380538f6a9a9305` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:28.676591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4faf93fcafba4617b380538f6a9a9305)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:28.678502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:28.732518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4faf93fcafba4617b380538f6a9a9305', 'request_id': 'req-942c1caa-0923-4297-adcb-d7f3b78c40bd'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:28.742265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4faf93fcafba4617b380538f6a9a9305', 'request_id': 'req-942c1caa-0923-4297-adcb-d7f3b78c40bd'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:28.749125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4faf93fcafba4617b380538f6a9a9305', 'request_id': 'req-942c1caa-0923-4297-adcb-d7f3b78c40bd'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:28.761886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] The token cache is being invalidate because project 4faf93fcafba4617b380538f6a9a9305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:57:28.762176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:28.762857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-942c1caa-0923-4297-adcb-d7f3b78c40bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:28.764434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1160/2321] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:57:28 2022] DELETE /identity/v3/projects/4faf93fcafba4617b380538f6a9a9305 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:30.156235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55457e78-116c-485d-818d-d35d323b6a6d service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:30.157727 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-55457e78-116c-485d-818d-d35d323b6a6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:30.171097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55457e78-116c-485d-818d-d35d323b6a6d service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:30.234925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:30.235805 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:30.236119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:30.236448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] PATH_INFO: `/v3/users/cd4f05aa641f47f5b1f5ff41ca984a10` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:30.237769 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd4f05aa641f47f5b1f5ff41ca984a10)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:30.239603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:30.305905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd4f05aa641f47f5b1f5ff41ca984a10'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:30.310853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:30.322534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd4f05aa641f47f5b1f5ff41ca984a10'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:30.323427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd4f05aa641f47f5b1f5ff41ca984a10'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:30.342108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd4f05aa641f47f5b1f5ff41ca984a10'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:30.355076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd4f05aa641f47f5b1f5ff41ca984a10'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:30.365563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1162/2322] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:57:30 2022] DELETE /identity/v3/users/cd4f05aa641f47f5b1f5ff41ca984a10 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:57:30.373444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65716608-60e8-4ff3-8bfe-a74d9767676a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:30.373988 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65716608-60e8-4ff3-8bfe-a74d9767676a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:30.374433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65716608-60e8-4ff3-8bfe-a74d9767676a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:30.405965 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-65716608-60e8-4ff3-8bfe-a74d9767676a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:30.408052 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-65716608-60e8-4ff3-8bfe-a74d9767676a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:30.416505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:30.477995 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1161/2323] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:57:30.488312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:30.489811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:30.553533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c549c-1102-48e0-86f7-7273dd857330 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:30.557818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14311660-3291-40d2-8c8c-d71f9c7ad603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:30.558484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14311660-3291-40d2-8c8c-d71f9c7ad603 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:30.558769 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14311660-3291-40d2-8c8c-d71f9c7ad603 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:30.559051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14311660-3291-40d2-8c8c-d71f9c7ad603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:30.570287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14311660-3291-40d2-8c8c-d71f9c7ad603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:30.571738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14311660-3291-40d2-8c8c-d71f9c7ad603 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:30.582960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1163/2324] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:57:31.189907 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65716608-60e8-4ff3-8bfe-a74d9767676a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:31.191433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-65716608-60e8-4ff3-8bfe-a74d9767676a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:31.210220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65716608-60e8-4ff3-8bfe-a74d9767676a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:31.218450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:31.219888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:31.222553 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:31.222875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] PATH_INFO: `/v3/projects/4ad611122b054a2eb44fc11b323b6c5f` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:31.226782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad611122b054a2eb44fc11b323b6c5f)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:31.228835 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:31.297411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad611122b054a2eb44fc11b323b6c5f', 'request_id': 'req-5af9faf7-8222-407d-acc3-a8fef62ae9bb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:31.306319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad611122b054a2eb44fc11b323b6c5f', 'request_id': 'req-5af9faf7-8222-407d-acc3-a8fef62ae9bb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:31.314940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad611122b054a2eb44fc11b323b6c5f', 'request_id': 'req-5af9faf7-8222-407d-acc3-a8fef62ae9bb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:31.328853 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] The token cache is being invalidate because project 4ad611122b054a2eb44fc11b323b6c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:57:31.329144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:31.330144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5af9faf7-8222-407d-acc3-a8fef62ae9bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:31.332087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1162/2325] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:57:31 2022] DELETE /identity/v3/projects/4ad611122b054a2eb44fc11b323b6c5f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:40.578834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=38 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:41.325779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:57:59.712541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14311660-3291-40d2-8c8c-d71f9c7ad603 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:57:59.714396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-14311660-3291-40d2-8c8c-d71f9c7ad603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:59.728899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14311660-3291-40d2-8c8c-d71f9c7ad603 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:57:59.794221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:57:59.795147 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:59.795462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:59.795743 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] PATH_INFO: `/v3/users/4b7276b3a0914806881c04495ed1ad6b` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:59.796976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b7276b3a0914806881c04495ed1ad6b)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:57:59.798911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:57:59.856104 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b7276b3a0914806881c04495ed1ad6b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:59.865732 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b7276b3a0914806881c04495ed1ad6b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:59.866478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b7276b3a0914806881c04495ed1ad6b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:59.881210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b7276b3a0914806881c04495ed1ad6b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:59.889786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b7276b3a0914806881c04495ed1ad6b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:57:59.896126 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1164/2326] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:57:59 2022] DELETE /identity/v3/users/4b7276b3a0914806881c04495ed1ad6b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:57:59.901002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973fbf-4cb2-46c7-8729-c25da1c2706c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:57:59.901525 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973fbf-4cb2-46c7-8729-c25da1c2706c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:57:59.901637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973fbf-4cb2-46c7-8729-c25da1c2706c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:57:59.921246 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-df973fbf-4cb2-46c7-8729-c25da1c2706c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:57:59.922696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-df973fbf-4cb2-46c7-8729-c25da1c2706c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:57:59.995659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1163/2327] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:58:00.006409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:00.007885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:00.079553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e82286-9031-4e29-89db-342ebc572503 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:00.083934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a716444-5e11-4b57-a00b-8cdece33aff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:00.084796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a716444-5e11-4b57-a00b-8cdece33aff9 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:00.085091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a716444-5e11-4b57-a00b-8cdece33aff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:00.085468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a716444-5e11-4b57-a00b-8cdece33aff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:00.097770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a716444-5e11-4b57-a00b-8cdece33aff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:00.099230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a716444-5e11-4b57-a00b-8cdece33aff9 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:00.111024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1165/2328] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:58:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:58:00.442474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df973fbf-4cb2-46c7-8729-c25da1c2706c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:00.449483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-df973fbf-4cb2-46c7-8729-c25da1c2706c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:00.473072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df973fbf-4cb2-46c7-8729-c25da1c2706c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:00.482578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:00.483702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:00.483844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:00.484122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] PATH_INFO: `/v3/projects/64e3949ba6b3406192340a338383947c` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:00.486052 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e3949ba6b3406192340a338383947c)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:00.488018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:00.540844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e3949ba6b3406192340a338383947c', 'request_id': 'req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:58:00.548745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e3949ba6b3406192340a338383947c', 'request_id': 'req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:58:00.555335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e3949ba6b3406192340a338383947c', 'request_id': 'req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:58:00.566673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] The token cache is being invalidate because project 64e3949ba6b3406192340a338383947c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:58:00.566963 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:58:00.567602 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fd801a80-6d66-4bc1-8d21-eab3c0b9f7a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:58:00.569161 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1164/2329] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:58:00 2022] DELETE /identity/v3/projects/64e3949ba6b3406192340a338383947c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:58:00.574134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e5129-a685-4599-8233-79a3587a34f3 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:00.574468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e5129-a685-4599-8233-79a3587a34f3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:00.574755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e5129-a685-4599-8233-79a3587a34f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:00.595890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-d77e5129-a685-4599-8233-79a3587a34f3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:58:00.597165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d77e5129-a685-4599-8233-79a3587a34f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:00.616415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:00.676282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1166/2330] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:58:00.685803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ae052-280a-482e-ae2a-96dde419d18d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:00.686456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ae052-280a-482e-ae2a-96dde419d18d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:00.686926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ae052-280a-482e-ae2a-96dde419d18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:00.709170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-060ae052-280a-482e-ae2a-96dde419d18d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:58:00.710402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-060ae052-280a-482e-ae2a-96dde419d18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:00.719146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1165/2331] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:58:00.723296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e5129-a685-4599-8233-79a3587a34f3 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:00.738438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e5129-a685-4599-8233-79a3587a34f3 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:00.744792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f26ad6-eb79-41b5-a1bb-d2431314392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:00.745501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f26ad6-eb79-41b5-a1bb-d2431314392a admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:00.745766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f26ad6-eb79-41b5-a1bb-d2431314392a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:00.746068 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f26ad6-eb79-41b5-a1bb-d2431314392a admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:00.747636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f26ad6-eb79-41b5-a1bb-d2431314392a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:58:00.747936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f26ad6-eb79-41b5-a1bb-d2431314392a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:00.750253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f26ad6-eb79-41b5-a1bb-d2431314392a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:00.764532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1167/2332] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:58:00.767264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ae052-280a-482e-ae2a-96dde419d18d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:00.778210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ae052-280a-482e-ae2a-96dde419d18d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:00.783214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc86488-e736-472e-ad85-22111f354989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:00.783985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc86488-e736-472e-ad85-22111f354989 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:00.784293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc86488-e736-472e-ad85-22111f354989 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:00.784850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc86488-e736-472e-ad85-22111f354989 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:00.785531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc86488-e736-472e-ad85-22111f354989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:00.787443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc86488-e736-472e-ad85-22111f354989 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:00.822464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1166/2333] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:58:00 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:58:00.826669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f26ad6-eb79-41b5-a1bb-d2431314392a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:00.837161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f26ad6-eb79-41b5-a1bb-d2431314392a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:00.907151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557617b-0c36-4728-8867-d9304ac2f938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:00.907943 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557617b-0c36-4728-8867-d9304ac2f938 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:00.908308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557617b-0c36-4728-8867-d9304ac2f938 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:00.908701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557617b-0c36-4728-8867-d9304ac2f938 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:00.909665 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557617b-0c36-4728-8867-d9304ac2f938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:00.912060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557617b-0c36-4728-8867-d9304ac2f938 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:00.942266 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1168/2334] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:58:00 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:58:00.943590 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc86488-e736-472e-ad85-22111f354989 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:00.955099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc86488-e736-472e-ad85-22111f354989 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:00.960927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab98f83-7882-4333-a9bc-a0e4d287f4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:00.961693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab98f83-7882-4333-a9bc-a0e4d287f4fc admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:00.962049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab98f83-7882-4333-a9bc-a0e4d287f4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:00.962386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab98f83-7882-4333-a9bc-a0e4d287f4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:00.963208 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab98f83-7882-4333-a9bc-a0e4d287f4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:00.965640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab98f83-7882-4333-a9bc-a0e4d287f4fc admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:00.980510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1167/2335] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 20:58:00.982649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557617b-0c36-4728-8867-d9304ac2f938 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:00.994296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557617b-0c36-4728-8867-d9304ac2f938 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:00.999174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a933c56-2cd4-4b3c-b373-1beb1f764177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:00.999850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a933c56-2cd4-4b3c-b373-1beb1f764177 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:01.000551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a933c56-2cd4-4b3c-b373-1beb1f764177 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:01.000551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a933c56-2cd4-4b3c-b373-1beb1f764177 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:01.001196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a933c56-2cd4-4b3c-b373-1beb1f764177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:01.002811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a933c56-2cd4-4b3c-b373-1beb1f764177 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:01.014506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1169/2336] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:58:01.019050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab98f83-7882-4333-a9bc-a0e4d287f4fc admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:01.029918 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab98f83-7882-4333-a9bc-a0e4d287f4fc admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:01.035507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6c638-71fd-4331-a537-d7b298746fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:01.037387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6c638-71fd-4331-a537-d7b298746fe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:01.037387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6c638-71fd-4331-a537-d7b298746fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:01.037387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6c638-71fd-4331-a537-d7b298746fe9 admin admin] PATH_INFO: `/v3/projects/e08c8f78187d474a865ea712616685cc/users/5634da4012cc4d468f84409736c28e09/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:01.054755 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6c638-71fd-4331-a537-d7b298746fe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e08c8f78187d474a865ea712616685cc, user_id=5634da4012cc4d468f84409736c28e09, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:01.057491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6c638-71fd-4331-a537-d7b298746fe9 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:01.072527 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1168/2337] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:58:01 2022] PUT /identity/v3/projects/e08c8f78187d474a865ea712616685cc/users/5634da4012cc4d468f84409736c28e09/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:58:01.077759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1d267-052d-4faf-8e0e-e47a69ca20f6 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:01.078047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1d267-052d-4faf-8e0e-e47a69ca20f6 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:01.078328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1d267-052d-4faf-8e0e-e47a69ca20f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:01.096639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-21d1d267-052d-4faf-8e0e-e47a69ca20f6 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:58:01.097843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-21d1d267-052d-4faf-8e0e-e47a69ca20f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:01.158456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1170/2338] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:58:01.168752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6c638-71fd-4331-a537-d7b298746fe9 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:01.170511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7ba6c638-71fd-4331-a537-d7b298746fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:01.193075 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:01.232883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6c638-71fd-4331-a537-d7b298746fe9 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:01.236461 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11953837-2ca6-43c6-92de-26003a714c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:01.237103 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11953837-2ca6-43c6-92de-26003a714c8c service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:01.237383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11953837-2ca6-43c6-92de-26003a714c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:01.237746 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11953837-2ca6-43c6-92de-26003a714c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:01.246565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11953837-2ca6-43c6-92de-26003a714c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:01.247984 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11953837-2ca6-43c6-92de-26003a714c8c service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:01.257855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1169/2339] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:58:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:58:02.935733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f3e46-f988-4e65-b10d-210eadc2793e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:02.936219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f3e46-f988-4e65-b10d-210eadc2793e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:02.936311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f3e46-f988-4e65-b10d-210eadc2793e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:03.095837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-2c8f3e46-f988-4e65-b10d-210eadc2793e None None] MFA Rules not processed for user `72aaaf0bf8ee461c83350055d207951e`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:58:03.097105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2c8f3e46-f988-4e65-b10d-210eadc2793e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:03.125390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:03.161379 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1171/2340] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:58:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:58:07.701874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee92c7-6b55-4597-9373-e0911d705358 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:07.702541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee92c7-6b55-4597-9373-e0911d705358 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:07.702541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee92c7-6b55-4597-9373-e0911d705358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:07.749470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-9cee92c7-6b55-4597-9373-e0911d705358 None None] MFA Rules not processed for user `5634da4012cc4d468f84409736c28e09`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:58:07.751038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-9cee92c7-6b55-4597-9373-e0911d705358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:07.836198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1170/2341] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:58:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:58:07.846202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f3e46-f988-4e65-b10d-210eadc2793e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:07.863657 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f3e46-f988-4e65-b10d-210eadc2793e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:07.869172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513cba3-ca82-4f9c-9816-e35f5bd9921e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:07.869928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513cba3-ca82-4f9c-9816-e35f5bd9921e admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:07.870448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513cba3-ca82-4f9c-9816-e35f5bd9921e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:07.870549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513cba3-ca82-4f9c-9816-e35f5bd9921e admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:07.871451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513cba3-ca82-4f9c-9816-e35f5bd9921e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:07.874109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513cba3-ca82-4f9c-9816-e35f5bd9921e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:07.905987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1172/2342] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:58:07 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:58:07.910130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee92c7-6b55-4597-9373-e0911d705358 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:07.922630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee92c7-6b55-4597-9373-e0911d705358 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:08.013649 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f28cad-9874-4bce-aade-59d4d174a28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:08.014335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f28cad-9874-4bce-aade-59d4d174a28e admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:08.014621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f28cad-9874-4bce-aade-59d4d174a28e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:08.014907 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f28cad-9874-4bce-aade-59d4d174a28e admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:08.015614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f28cad-9874-4bce-aade-59d4d174a28e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:08.017648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f28cad-9874-4bce-aade-59d4d174a28e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:08.044396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1171/2343] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:58:07 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 20:58:08.048841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513cba3-ca82-4f9c-9816-e35f5bd9921e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:08.068369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513cba3-ca82-4f9c-9816-e35f5bd9921e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:08.077389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7c74c-888e-4205-9fca-38a8c30c04c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:08.078686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7c74c-888e-4205-9fca-38a8c30c04c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:08.079135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7c74c-888e-4205-9fca-38a8c30c04c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:08.079782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7c74c-888e-4205-9fca-38a8c30c04c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:08.080866 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7c74c-888e-4205-9fca-38a8c30c04c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:08.082719 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7c74c-888e-4205-9fca-38a8c30c04c8 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:08.099796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1173/2344] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:58:08.105123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f28cad-9874-4bce-aade-59d4d174a28e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:08.121782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f28cad-9874-4bce-aade-59d4d174a28e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:08.128691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a793f7-38ef-4362-8926-a978b0c2fe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:08.132378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a793f7-38ef-4362-8926-a978b0c2fe94 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:08.132378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a793f7-38ef-4362-8926-a978b0c2fe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:08.132378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a793f7-38ef-4362-8926-a978b0c2fe94 admin admin] PATH_INFO: `/v3/projects/1bd1c6a8ce4744f88fe546937f5c8c46/users/5bad2a3e3d744290901be1c2227ba50f/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:08.148058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a793f7-38ef-4362-8926-a978b0c2fe94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd1c6a8ce4744f88fe546937f5c8c46, user_id=5bad2a3e3d744290901be1c2227ba50f, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:08.150838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a793f7-38ef-4362-8926-a978b0c2fe94 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:08.166753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1172/2345] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:58:08 2022] PUT /identity/v3/projects/1bd1c6a8ce4744f88fe546937f5c8c46/users/5bad2a3e3d744290901be1c2227ba50f/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:58:08.170949 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7c74c-888e-4205-9fca-38a8c30c04c8 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:08.181712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7c74c-888e-4205-9fca-38a8c30c04c8 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:08.249797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d807147-98f5-420f-a87f-585d0710b00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:08.250484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d807147-98f5-420f-a87f-585d0710b00b admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:08.250770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d807147-98f5-420f-a87f-585d0710b00b admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:08.251052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d807147-98f5-420f-a87f-585d0710b00b admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:08.251981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d807147-98f5-420f-a87f-585d0710b00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:08.253465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d807147-98f5-420f-a87f-585d0710b00b admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:08.265114 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1174/2346] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:58:08.269072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a793f7-38ef-4362-8926-a978b0c2fe94 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:08.280798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a793f7-38ef-4362-8926-a978b0c2fe94 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:08.286175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a437cf-db4f-4371-8cad-f9b5b42ed043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:08.287078 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a437cf-db4f-4371-8cad-f9b5b42ed043 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:08.287377 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a437cf-db4f-4371-8cad-f9b5b42ed043 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:08.287654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a437cf-db4f-4371-8cad-f9b5b42ed043 admin admin] PATH_INFO: `/v3/projects/1bd1c6a8ce4744f88fe546937f5c8c46/users/5bad2a3e3d744290901be1c2227ba50f/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:08.289784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a437cf-db4f-4371-8cad-f9b5b42ed043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd1c6a8ce4744f88fe546937f5c8c46, user_id=5bad2a3e3d744290901be1c2227ba50f, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:08.292547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a437cf-db4f-4371-8cad-f9b5b42ed043 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:08.305870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1173/2347] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:58:08 2022] PUT /identity/v3/projects/1bd1c6a8ce4744f88fe546937f5c8c46/users/5bad2a3e3d744290901be1c2227ba50f/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:58:14.977481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dece739-bcd0-46fe-ba9a-9341724e7142 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:14.978033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dece739-bcd0-46fe-ba9a-9341724e7142 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:14.978151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dece739-bcd0-46fe-ba9a-9341724e7142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:15.027648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-4dece739-bcd0-46fe-ba9a-9341724e7142 None None] MFA Rules not processed for user `5bad2a3e3d744290901be1c2227ba50f`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:58:15.030984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4dece739-bcd0-46fe-ba9a-9341724e7142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:15.153134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1175/2348] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:58:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:58:15.173364 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a437cf-db4f-4371-8cad-f9b5b42ed043 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:15.175109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b3a437cf-db4f-4371-8cad-f9b5b42ed043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:15.191013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:15.284868 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a437cf-db4f-4371-8cad-f9b5b42ed043 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:15.291610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ce19e-b325-4940-8ee3-c5fcf1157eb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:15.296998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ce19e-b325-4940-8ee3-c5fcf1157eb3 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:15.296998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ce19e-b325-4940-8ee3-c5fcf1157eb3 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:15.296998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ce19e-b325-4940-8ee3-c5fcf1157eb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:15.365770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ce19e-b325-4940-8ee3-c5fcf1157eb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:15.368618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ce19e-b325-4940-8ee3-c5fcf1157eb3 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:15.383115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1174/2349] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:58:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:58:18.441152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ee284-3c31-4ae3-b7ac-49b9a176f897 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:18.441152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ee284-3c31-4ae3-b7ac-49b9a176f897 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:18.441152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ee284-3c31-4ae3-b7ac-49b9a176f897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:18.465850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-662ee284-3c31-4ae3-b7ac-49b9a176f897 None None] MFA Rules not processed for user `483b27bfe5674a528c29a15a054cc617`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:58:18.467322 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-662ee284-3c31-4ae3-b7ac-49b9a176f897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:18.493423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:18.548587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1176/2350] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:58:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:58:25.377152 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=54 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:28.537941 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=15 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:43.387371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc01993-be56-4689-a4fd-ac6fed7379af None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:43.387371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc01993-be56-4689-a4fd-ac6fed7379af None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:43.387371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc01993-be56-4689-a4fd-ac6fed7379af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:43.404558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-bcc01993-be56-4689-a4fd-ac6fed7379af None None] MFA Rules not processed for user `5634da4012cc4d468f84409736c28e09`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:58:43.405785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bcc01993-be56-4689-a4fd-ac6fed7379af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:43.418970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1175/2351] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:58:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:58:49.365580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831d510-080b-430f-9a8c-2755e24f0021 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:49.366110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831d510-080b-430f-9a8c-2755e24f0021 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:49.366591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831d510-080b-430f-9a8c-2755e24f0021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:49.388516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-9831d510-080b-430f-9a8c-2755e24f0021 None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:58:49.389754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9831d510-080b-430f-9a8c-2755e24f0021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:49.461798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1177/2352] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 20:58:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:58:49.522641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc01993-be56-4689-a4fd-ac6fed7379af None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:58:49.524681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bcc01993-be56-4689-a4fd-ac6fed7379af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:58:49.611425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc01993-be56-4689-a4fd-ac6fed7379af None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:58:49.631730 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e7f4d-038a-4e61-8096-fb065b708205 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:58:49.632438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e7f4d-038a-4e61-8096-fb065b708205 service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:58:49.632731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e7f4d-038a-4e61-8096-fb065b708205 service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:58:49.632997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e7f4d-038a-4e61-8096-fb065b708205 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:58:49.649971 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e7f4d-038a-4e61-8096-fb065b708205 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:58:49.652303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e7f4d-038a-4e61-8096-fb065b708205 service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:58:49.662556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1176/2353] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 20:58:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:58:59.453476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=25 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:58:59.658410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=35 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:04.606079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29f78e-6440-48bb-822d-5fdd6460de96 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:04.607240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29f78e-6440-48bb-822d-5fdd6460de96 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:04.607533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29f78e-6440-48bb-822d-5fdd6460de96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:04.635493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-1f29f78e-6440-48bb-822d-5fdd6460de96 None None] MFA Rules not processed for user `ac1b24e50c894dcf8e887275fd91a174`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:04.638276 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1f29f78e-6440-48bb-822d-5fdd6460de96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:04.739450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1178/2354] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:14.022042 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e22d7-0d72-4f0e-9691-5775580bfd5f None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:14.022594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e22d7-0d72-4f0e-9691-5775580bfd5f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:14.022594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e22d7-0d72-4f0e-9691-5775580bfd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:14.041105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-257e22d7-0d72-4f0e-9691-5775580bfd5f None None] MFA Rules not processed for user `d4c6ceff33ee449193fe6d4f7b2c01ec`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:14.042707 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-257e22d7-0d72-4f0e-9691-5775580bfd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:14.107342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1177/2355] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:59:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:14.178148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924181a0-585b-457d-8597-7693c6c41543 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:14.179003 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924181a0-585b-457d-8597-7693c6c41543 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:14.179502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924181a0-585b-457d-8597-7693c6c41543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:14.212397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-924181a0-585b-457d-8597-7693c6c41543 None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:14.214726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-924181a0-585b-457d-8597-7693c6c41543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:14.222379 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1179/2356] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jul 19 20:59:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:14.236260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e22d7-0d72-4f0e-9691-5775580bfd5f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:14.250630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e22d7-0d72-4f0e-9691-5775580bfd5f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:14.257386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251d8f1-1bb1-4225-8836-45e47f7c3d23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:14.258361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251d8f1-1bb1-4225-8836-45e47f7c3d23 service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:14.258906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251d8f1-1bb1-4225-8836-45e47f7c3d23 service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:14.259398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251d8f1-1bb1-4225-8836-45e47f7c3d23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:14.270246 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251d8f1-1bb1-4225-8836-45e47f7c3d23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:14.272781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251d8f1-1bb1-4225-8836-45e47f7c3d23 service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:14.282014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1178/2357] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 20:59:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:59:16.864029 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924181a0-585b-457d-8597-7693c6c41543 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:16.866275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-924181a0-585b-457d-8597-7693c6c41543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:16.941160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924181a0-585b-457d-8597-7693c6c41543 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:16.946289 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ab4a0-1ee0-4aeb-91ba-8b719b5cbe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:16.947241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab4a0-1ee0-4aeb-91ba-8b719b5cbe10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:16.947742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab4a0-1ee0-4aeb-91ba-8b719b5cbe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:16.948023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ab4a0-1ee0-4aeb-91ba-8b719b5cbe10 admin admin] PATH_INFO: `/v3/users/72aaaf0bf8ee461c83350055d207951e` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:16.949357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ab4a0-1ee0-4aeb-91ba-8b719b5cbe10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72aaaf0bf8ee461c83350055d207951e)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:16.951275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ab4a0-1ee0-4aeb-91ba-8b719b5cbe10 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:17.001747 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7c9ab4a0-1ee0-4aeb-91ba-8b719b5cbe10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72aaaf0bf8ee461c83350055d207951e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.012703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7c9ab4a0-1ee0-4aeb-91ba-8b719b5cbe10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72aaaf0bf8ee461c83350055d207951e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.013380 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7c9ab4a0-1ee0-4aeb-91ba-8b719b5cbe10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72aaaf0bf8ee461c83350055d207951e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.026496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7c9ab4a0-1ee0-4aeb-91ba-8b719b5cbe10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72aaaf0bf8ee461c83350055d207951e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.033197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7c9ab4a0-1ee0-4aeb-91ba-8b719b5cbe10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72aaaf0bf8ee461c83350055d207951e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.039036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1180/2358] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:59:16 2022] DELETE /identity/v3/users/72aaaf0bf8ee461c83350055d207951e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:17.214491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251d8f1-1bb1-4225-8836-45e47f7c3d23 service swift] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:17.227385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251d8f1-1bb1-4225-8836-45e47f7c3d23 service swift] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:17.296783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:17.297666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.297948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.298226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] PATH_INFO: `/v3/projects/051e2bfc96334054956366e1308d067d` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.299425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=051e2bfc96334054956366e1308d067d)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:17.301456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:17.353028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '051e2bfc96334054956366e1308d067d', 'request_id': 'req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.361317 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '051e2bfc96334054956366e1308d067d', 'request_id': 'req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.368638 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '051e2bfc96334054956366e1308d067d', 'request_id': 'req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.379617 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] The token cache is being invalidate because project 051e2bfc96334054956366e1308d067d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:59:17.379912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.380687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-46ccfaa5-ff9a-4a80-a48e-0c3bc7abf971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:17.382335 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1179/2359] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:59:17 2022] DELETE /identity/v3/projects/051e2bfc96334054956366e1308d067d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:17.390196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b139d2-59d1-4f69-a97c-e79e5c07b603 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.390624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b139d2-59d1-4f69-a97c-e79e5c07b603 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.390902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b139d2-59d1-4f69-a97c-e79e5c07b603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.408933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-24b139d2-59d1-4f69-a97c-e79e5c07b603 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:17.410099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-24b139d2-59d1-4f69-a97c-e79e5c07b603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:17.422864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:17.472318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1181/2360] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 20:59:17.480589 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ba2b8-e3aa-4d66-a5aa-4f9b68bc7e2f None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.480807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ba2b8-e3aa-4d66-a5aa-4f9b68bc7e2f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.481083 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ba2b8-e3aa-4d66-a5aa-4f9b68bc7e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.502197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-7c8ba2b8-e3aa-4d66-a5aa-4f9b68bc7e2f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:17.503402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7c8ba2b8-e3aa-4d66-a5aa-4f9b68bc7e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:17.512126 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1180/2361] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:17.516448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b139d2-59d1-4f69-a97c-e79e5c07b603 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:17.526146 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b139d2-59d1-4f69-a97c-e79e5c07b603 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:17.531340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33db0c-6f58-48ff-85f5-6831c14044c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:17.531848 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33db0c-6f58-48ff-85f5-6831c14044c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.532128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33db0c-6f58-48ff-85f5-6831c14044c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.532442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33db0c-6f58-48ff-85f5-6831c14044c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.533171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33db0c-6f58-48ff-85f5-6831c14044c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 20:59:17.533477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33db0c-6f58-48ff-85f5-6831c14044c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:17.535056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33db0c-6f58-48ff-85f5-6831c14044c9 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:17.549192 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1182/2362] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 20:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 20:59:17.553499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ba2b8-e3aa-4d66-a5aa-4f9b68bc7e2f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:17.564147 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ba2b8-e3aa-4d66-a5aa-4f9b68bc7e2f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:17.569515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143aa4a-7a11-44c4-8fa9-a9d9a7f19a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:17.570171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143aa4a-7a11-44c4-8fa9-a9d9a7f19a20 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.570455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143aa4a-7a11-44c4-8fa9-a9d9a7f19a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.570729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143aa4a-7a11-44c4-8fa9-a9d9a7f19a20 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.571418 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143aa4a-7a11-44c4-8fa9-a9d9a7f19a20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:17.574127 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143aa4a-7a11-44c4-8fa9-a9d9a7f19a20 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:17.588044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:17.606843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1181/2363] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 20:59:17 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:17.608008 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33db0c-6f58-48ff-85f5-6831c14044c9 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:17.618231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33db0c-6f58-48ff-85f5-6831c14044c9 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:17.676997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644086a4-0270-41a4-84bb-b73086aaa69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:17.677667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644086a4-0270-41a4-84bb-b73086aaa69c admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.677949 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644086a4-0270-41a4-84bb-b73086aaa69c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.678223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644086a4-0270-41a4-84bb-b73086aaa69c admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.678911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644086a4-0270-41a4-84bb-b73086aaa69c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:17.680838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644086a4-0270-41a4-84bb-b73086aaa69c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:17.705844 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1183/2364] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 20:59:17 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 20:59:17.718211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143aa4a-7a11-44c4-8fa9-a9d9a7f19a20 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:17.734927 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143aa4a-7a11-44c4-8fa9-a9d9a7f19a20 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:17.740912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e887a4-db6a-4954-827a-d206d6371d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:17.741598 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e887a4-db6a-4954-827a-d206d6371d38 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.742197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e887a4-db6a-4954-827a-d206d6371d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.742714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e887a4-db6a-4954-827a-d206d6371d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.743453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e887a4-db6a-4954-827a-d206d6371d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:17.745773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e887a4-db6a-4954-827a-d206d6371d38 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:17.762669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1182/2365] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:17.763850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644086a4-0270-41a4-84bb-b73086aaa69c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:17.775572 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644086a4-0270-41a4-84bb-b73086aaa69c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:17.783306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f99fe7-8ed3-4cb0-8593-9063f28c0a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:17.784204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f99fe7-8ed3-4cb0-8593-9063f28c0a78 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.784804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f99fe7-8ed3-4cb0-8593-9063f28c0a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.785246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f99fe7-8ed3-4cb0-8593-9063f28c0a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.785992 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f99fe7-8ed3-4cb0-8593-9063f28c0a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:17.787662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f99fe7-8ed3-4cb0-8593-9063f28c0a78 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:17.800228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1184/2366] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 20:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 20:59:17.805030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e887a4-db6a-4954-827a-d206d6371d38 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:17.816960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e887a4-db6a-4954-827a-d206d6371d38 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:17.822334 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec332e-a2b4-4f56-810e-d4efd39f08b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:17.823328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec332e-a2b4-4f56-810e-d4efd39f08b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.823687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec332e-a2b4-4f56-810e-d4efd39f08b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.824027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec332e-a2b4-4f56-810e-d4efd39f08b5 admin admin] PATH_INFO: `/v3/projects/fc584a9767e74858979d7963227d7ada/users/a42ee6d51d5149ba91477623a358e26a/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.841809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec332e-a2b4-4f56-810e-d4efd39f08b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc584a9767e74858979d7963227d7ada, user_id=a42ee6d51d5149ba91477623a358e26a, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:17.844636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec332e-a2b4-4f56-810e-d4efd39f08b5 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:17.861844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1183/2367] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 20:59:17 2022] PUT /identity/v3/projects/fc584a9767e74858979d7963227d7ada/users/a42ee6d51d5149ba91477623a358e26a/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 20:59:17.873911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500428-0136-4688-97d4-e8aed6b77ccc None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:17.874493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500428-0136-4688-97d4-e8aed6b77ccc None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:17.874493 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500428-0136-4688-97d4-e8aed6b77ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:17.919762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-66500428-0136-4688-97d4-e8aed6b77ccc None None] MFA Rules not processed for user `a42ee6d51d5149ba91477623a358e26a`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:17.919762 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-66500428-0136-4688-97d4-e8aed6b77ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:17.996944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1185/2368] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:59:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:18.026493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec332e-a2b4-4f56-810e-d4efd39f08b5 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:18.027629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-90ec332e-a2b4-4f56-810e-d4efd39f08b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:18.111359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec332e-a2b4-4f56-810e-d4efd39f08b5 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:18.116667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dc8ff-c54a-47f1-b5bc-e4d5a9f6f621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:18.117570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dc8ff-c54a-47f1-b5bc-e4d5a9f6f621 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:18.117852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dc8ff-c54a-47f1-b5bc-e4d5a9f6f621 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:18.118130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dc8ff-c54a-47f1-b5bc-e4d5a9f6f621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:18.132241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dc8ff-c54a-47f1-b5bc-e4d5a9f6f621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:18.133771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dc8ff-c54a-47f1-b5bc-e4d5a9f6f621 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:18.144629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1184/2369] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:59:19.584205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ee205-c0e5-4f44-8590-fad7e01c7848 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:19.584779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ee205-c0e5-4f44-8590-fad7e01c7848 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:19.584875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ee205-c0e5-4f44-8590-fad7e01c7848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:19.603173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-243ee205-c0e5-4f44-8590-fad7e01c7848 None None] MFA Rules not processed for user `a42ee6d51d5149ba91477623a358e26a`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:19.605173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-243ee205-c0e5-4f44-8590-fad7e01c7848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:19.612313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1186/2370] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:59:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:21.096153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dc8ff-c54a-47f1-b5bc-e4d5a9f6f621 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:21.097676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-761dc8ff-c54a-47f1-b5bc-e4d5a9f6f621 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:21.158734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dc8ff-c54a-47f1-b5bc-e4d5a9f6f621 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:21.162750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49065d9d-d711-41da-bdf0-86f0a9dda6af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:21.163408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49065d9d-d711-41da-bdf0-86f0a9dda6af service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:21.163820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49065d9d-d711-41da-bdf0-86f0a9dda6af service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:21.164108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49065d9d-d711-41da-bdf0-86f0a9dda6af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:21.165997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-49065d9d-d711-41da-bdf0-86f0a9dda6af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:21.204201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ee205-c0e5-4f44-8590-fad7e01c7848 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:21.214393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ee205-c0e5-4f44-8590-fad7e01c7848 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:21.218189 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b5229-b808-4258-a11b-4f33327069c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:21.218846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b5229-b808-4258-a11b-4f33327069c2 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:21.219125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b5229-b808-4258-a11b-4f33327069c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:21.219420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b5229-b808-4258-a11b-4f33327069c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:21.221186 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7b1b5229-b808-4258-a11b-4f33327069c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:21.226269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:21.262336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49065d9d-d711-41da-bdf0-86f0a9dda6af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:21.263799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49065d9d-d711-41da-bdf0-86f0a9dda6af service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:21.276916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1185/2371] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:21.309022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b5229-b808-4258-a11b-4f33327069c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:21.310454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b5229-b808-4258-a11b-4f33327069c2 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:21.323393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1187/2372] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 20:59:23.223541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49065d9d-d711-41da-bdf0-86f0a9dda6af service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:23.227859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:23.234434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49065d9d-d711-41da-bdf0-86f0a9dda6af service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:23.238721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a3800-3eb4-482c-8499-f7f291e8a409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:23.239369 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a3800-3eb4-482c-8499-f7f291e8a409 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:23.239648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a3800-3eb4-482c-8499-f7f291e8a409 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:23.239922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a3800-3eb4-482c-8499-f7f291e8a409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:23.241767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fd4a3800-3eb4-482c-8499-f7f291e8a409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:23.309727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a3800-3eb4-482c-8499-f7f291e8a409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:23.311182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a3800-3eb4-482c-8499-f7f291e8a409 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:23.321992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1186/2373] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:31.266581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b5229-b808-4258-a11b-4f33327069c2 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:31.268199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7b1b5229-b808-4258-a11b-4f33327069c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:31.282963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b5229-b808-4258-a11b-4f33327069c2 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:31.288564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ceec1-ac71-4b0a-b07a-566a3d06995c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:31.289214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ceec1-ac71-4b0a-b07a-566a3d06995c service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:31.289498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ceec1-ac71-4b0a-b07a-566a3d06995c service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:31.289773 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ceec1-ac71-4b0a-b07a-566a3d06995c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:31.291925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-157ceec1-ac71-4b0a-b07a-566a3d06995c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:31.358501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ceec1-ac71-4b0a-b07a-566a3d06995c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:31.360106 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ceec1-ac71-4b0a-b07a-566a3d06995c service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:31.369755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1188/2374] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 20:59:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:33.317842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=29 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:37.868662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a3800-3eb4-482c-8499-f7f291e8a409 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:37.878626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a3800-3eb4-482c-8499-f7f291e8a409 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:37.882529 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbbea3-30f4-4652-9100-707a985e5fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:37.883504 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbbea3-30f4-4652-9100-707a985e5fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:37.884004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbbea3-30f4-4652-9100-707a985e5fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:37.884442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbbea3-30f4-4652-9100-707a985e5fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:37.886694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-55fbbea3-30f4-4652-9100-707a985e5fe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:37.967085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbbea3-30f4-4652-9100-707a985e5fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:37.968553 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fbbea3-30f4-4652-9100-707a985e5fe9 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:37.978202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1187/2375] 10.222.0.85 () {60 vars in 1396 bytes} [Tue Jul 19 20:59:37 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 20:59:41.365274 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=59 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:44.292404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d273c39-2c56-437b-85d9-64a70124f1cb None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:44.292998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d273c39-2c56-437b-85d9-64a70124f1cb None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:44.292998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d273c39-2c56-437b-85d9-64a70124f1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:44.322750 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0d273c39-2c56-437b-85d9-64a70124f1cb None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:44.323684 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0d273c39-2c56-437b-85d9-64a70124f1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:44.391070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1189/2376] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:44.407184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbbea3-30f4-4652-9100-707a985e5fe9 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:44.409423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-55fbbea3-30f4-4652-9100-707a985e5fe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:44.506615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbbea3-30f4-4652-9100-707a985e5fe9 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:44.513580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995e46e-83b0-4b13-92a6-84612b8c3bd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:44.514503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995e46e-83b0-4b13-92a6-84612b8c3bd8 service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:44.514989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995e46e-83b0-4b13-92a6-84612b8c3bd8 service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:44.515450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995e46e-83b0-4b13-92a6-84612b8c3bd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:44.525397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995e46e-83b0-4b13-92a6-84612b8c3bd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:44.527096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3995e46e-83b0-4b13-92a6-84612b8c3bd8 service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:44.537012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1188/2377] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 20:59:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:59:54.382094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=25 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:54.532570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=60 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:55.293967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d273c39-2c56-437b-85d9-64a70124f1cb None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:55.295633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0d273c39-2c56-437b-85d9-64a70124f1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:55.383256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d273c39-2c56-437b-85d9-64a70124f1cb None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:55.393384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54e05f-cd87-4c42-85d5-69493e6b2630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:55.394561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54e05f-cd87-4c42-85d5-69493e6b2630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:55.395148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54e05f-cd87-4c42-85d5-69493e6b2630 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:55.395706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54e05f-cd87-4c42-85d5-69493e6b2630 admin admin] PATH_INFO: `/v3/users/483b27bfe5674a528c29a15a054cc617` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:55.397237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54e05f-cd87-4c42-85d5-69493e6b2630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=483b27bfe5674a528c29a15a054cc617)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:55.399570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54e05f-cd87-4c42-85d5-69493e6b2630 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:55.471719 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7d54e05f-cd87-4c42-85d5-69493e6b2630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '483b27bfe5674a528c29a15a054cc617'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:55.481214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7d54e05f-cd87-4c42-85d5-69493e6b2630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '483b27bfe5674a528c29a15a054cc617'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:55.481955 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7d54e05f-cd87-4c42-85d5-69493e6b2630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '483b27bfe5674a528c29a15a054cc617'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:55.496412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7d54e05f-cd87-4c42-85d5-69493e6b2630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '483b27bfe5674a528c29a15a054cc617'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:55.504097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7d54e05f-cd87-4c42-85d5-69493e6b2630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '483b27bfe5674a528c29a15a054cc617'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:55.511649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1190/2378] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:59:55 2022] DELETE /identity/v3/users/483b27bfe5674a528c29a15a054cc617 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:55.735712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995e46e-83b0-4b13-92a6-84612b8c3bd8 service swift] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:55.751403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995e46e-83b0-4b13-92a6-84612b8c3bd8 service swift] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:55.824054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:55.825422 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:55.826423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:55.826423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] PATH_INFO: `/v3/projects/72c814e2b46244db93e801a78763d880` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:55.829543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=72c814e2b46244db93e801a78763d880)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:55.833210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:55.882876 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72c814e2b46244db93e801a78763d880', 'request_id': 'req-00f97790-a9fb-4fb8-b735-df492a9f1a8e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:55.890865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72c814e2b46244db93e801a78763d880', 'request_id': 'req-00f97790-a9fb-4fb8-b735-df492a9f1a8e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:55.897356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72c814e2b46244db93e801a78763d880', 'request_id': 'req-00f97790-a9fb-4fb8-b735-df492a9f1a8e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:55.907772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] The token cache is being invalidate because project 72c814e2b46244db93e801a78763d880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 20:59:55.908062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:55.908736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:55.910460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1189/2379] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 20:59:55 2022] DELETE /identity/v3/projects/72c814e2b46244db93e801a78763d880 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:56.190176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40019d60-831c-4c85-bec6-820bca90821c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:56.190468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40019d60-831c-4c85-bec6-820bca90821c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:56.191004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40019d60-831c-4c85-bec6-820bca90821c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:56.209342 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-40019d60-831c-4c85-bec6-820bca90821c None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:56.210580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-40019d60-831c-4c85-bec6-820bca90821c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:56.268735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1191/2380] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 20:59:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:56.281039 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:56.282468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:56.354662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f97790-a9fb-4fb8-b735-df492a9f1a8e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:56.359755 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f480906-0961-4279-b2cd-09406f985f86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:56.360446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f480906-0961-4279-b2cd-09406f985f86 service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:56.360745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f480906-0961-4279-b2cd-09406f985f86 service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:56.361025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f480906-0961-4279-b2cd-09406f985f86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:56.369420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f480906-0961-4279-b2cd-09406f985f86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:56.371014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f480906-0961-4279-b2cd-09406f985f86 service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:56.379049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1190/2381] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 20:59:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:59:56.930855 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf1017-f05e-40b9-87e9-2eae29f6e948 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:56.930855 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf1017-f05e-40b9-87e9-2eae29f6e948 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:56.931995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf1017-f05e-40b9-87e9-2eae29f6e948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:56.951318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-b6cf1017-f05e-40b9-87e9-2eae29f6e948 None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:56.952740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b6cf1017-f05e-40b9-87e9-2eae29f6e948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:56.960201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1192/2382] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jul 19 20:59:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:56.987694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f480906-0961-4279-b2cd-09406f985f86 service swift] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:56.999742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f480906-0961-4279-b2cd-09406f985f86 service swift] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:57.004902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985180a2-b0c5-425d-b705-650a341ca68d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:57.005551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985180a2-b0c5-425d-b705-650a341ca68d service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:57.005834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985180a2-b0c5-425d-b705-650a341ca68d service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:57.006139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985180a2-b0c5-425d-b705-650a341ca68d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:57.015729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985180a2-b0c5-425d-b705-650a341ca68d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:57.017407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985180a2-b0c5-425d-b705-650a341ca68d service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:57.025966 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1191/2383] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 20:59:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 20:59:57.240114 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f940695-9263-48d6-ae2c-008b1ba07fec None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:57.240584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f940695-9263-48d6-ae2c-008b1ba07fec None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:57.240872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f940695-9263-48d6-ae2c-008b1ba07fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:57.259258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-7f940695-9263-48d6-ae2c-008b1ba07fec None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:57.260473 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7f940695-9263-48d6-ae2c-008b1ba07fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:57.268042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1193/2384] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 20:59:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:57.280436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985180a2-b0c5-425d-b705-650a341ca68d service swift] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:57.291497 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985180a2-b0c5-425d-b705-650a341ca68d service swift] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:57.296583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46734a75-0d7e-43f6-ae29-68ef068cb57f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:57.297220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46734a75-0d7e-43f6-ae29-68ef068cb57f service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:57.297504 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46734a75-0d7e-43f6-ae29-68ef068cb57f service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:57.297782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46734a75-0d7e-43f6-ae29-68ef068cb57f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:57.303036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:57.307800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46734a75-0d7e-43f6-ae29-68ef068cb57f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:57.309553 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46734a75-0d7e-43f6-ae29-68ef068cb57f service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:57.319960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1192/2385] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 20:59:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:59:57.656427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ac107-0894-4a55-87eb-1971010cb575 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:57.656726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ac107-0894-4a55-87eb-1971010cb575 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:57.657007 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ac107-0894-4a55-87eb-1971010cb575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:57.675097 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-f73ac107-0894-4a55-87eb-1971010cb575 None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 20:59:57.676309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f73ac107-0894-4a55-87eb-1971010cb575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:57.683547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1194/2386] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jul 19 20:59:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 20:59:57.696888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46734a75-0d7e-43f6-ae29-68ef068cb57f service swift] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:57.709019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46734a75-0d7e-43f6-ae29-68ef068cb57f service swift] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:57.715466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7e44a-f02c-43f2-b429-6a7e0e81af56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:57.716284 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7e44a-f02c-43f2-b429-6a7e0e81af56 service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:57.716513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7e44a-f02c-43f2-b429-6a7e0e81af56 service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:57.716803 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7e44a-f02c-43f2-b429-6a7e0e81af56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:57.726974 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7e44a-f02c-43f2-b429-6a7e0e81af56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:57.728631 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7e44a-f02c-43f2-b429-6a7e0e81af56 service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:57.739857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1193/2387] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 20:59:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 20:59:59.822874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ac107-0894-4a55-87eb-1971010cb575 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 20:59:59.824407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f73ac107-0894-4a55-87eb-1971010cb575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 20:59:59.830616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 20:59:59.892831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ac107-0894-4a55-87eb-1971010cb575 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 20:59:59.897602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 20:59:59.898477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:59.898764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:59.899044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] PATH_INFO: `/v3/users/a42ee6d51d5149ba91477623a358e26a` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 20:59:59.900273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a42ee6d51d5149ba91477623a358e26a)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 20:59:59.902230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 20:59:59.954291 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a42ee6d51d5149ba91477623a358e26a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:59.962720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a42ee6d51d5149ba91477623a358e26a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:59.963367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a42ee6d51d5149ba91477623a358e26a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:59.974997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a42ee6d51d5149ba91477623a358e26a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:59.982142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a42ee6d51d5149ba91477623a358e26a'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 20:59:59.988048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1195/2388] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 20:59:59 2022] DELETE /identity/v3/users/a42ee6d51d5149ba91477623a358e26a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 20:59:59.992846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bfabc-998f-45be-b25e-22bf049f2532 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 20:59:59.993887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bfabc-998f-45be-b25e-22bf049f2532 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 20:59:59.993983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bfabc-998f-45be-b25e-22bf049f2532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:00.011682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-408bfabc-998f-45be-b25e-22bf049f2532 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:00.012936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-408bfabc-998f-45be-b25e-22bf049f2532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:00.072902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1194/2389] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 20:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:00.082659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:00.084110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:00.143667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c244b1c-d4b4-49ad-bee3-17514a0b7a18 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:00.147324 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a88d0b-c2e5-4433-94f6-0ed01847de9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:00.147965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a88d0b-c2e5-4433-94f6-0ed01847de9f service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:00.148749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a88d0b-c2e5-4433-94f6-0ed01847de9f service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:00.148749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a88d0b-c2e5-4433-94f6-0ed01847de9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:00.158888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a88d0b-c2e5-4433-94f6-0ed01847de9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:00.160368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a88d0b-c2e5-4433-94f6-0ed01847de9f service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:00.170599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1196/2390] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:00.361348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bfabc-998f-45be-b25e-22bf049f2532 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:00.362843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-408bfabc-998f-45be-b25e-22bf049f2532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:00.376923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bfabc-998f-45be-b25e-22bf049f2532 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:00.386831 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:00.387935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:00.388408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:00.388871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] PATH_INFO: `/v3/projects/fc584a9767e74858979d7963227d7ada` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:00.390456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc584a9767e74858979d7963227d7ada)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:00.392920 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:00.479088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc584a9767e74858979d7963227d7ada', 'request_id': 'req-0426092d-e1b6-4c17-8470-12adf4384d2a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:00.491447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc584a9767e74858979d7963227d7ada', 'request_id': 'req-0426092d-e1b6-4c17-8470-12adf4384d2a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:00.508278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc584a9767e74858979d7963227d7ada', 'request_id': 'req-0426092d-e1b6-4c17-8470-12adf4384d2a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:00.523753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] The token cache is being invalidate because project fc584a9767e74858979d7963227d7ada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:00:00.524061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:00.525456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-0426092d-e1b6-4c17-8470-12adf4384d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:00:00.528372 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1195/2391] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:00:00 2022] DELETE /identity/v3/projects/fc584a9767e74858979d7963227d7ada => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:00:01.057684 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a88d0b-c2e5-4433-94f6-0ed01847de9f service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:01.066751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a88d0b-c2e5-4433-94f6-0ed01847de9f service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:01.090937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:01.119481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a766d69-c925-43f2-98ab-71b72b85ff22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:01.120133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a766d69-c925-43f2-98ab-71b72b85ff22 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:01.120456 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a766d69-c925-43f2-98ab-71b72b85ff22 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:01.120735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a766d69-c925-43f2-98ab-71b72b85ff22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:01.122544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1a766d69-c925-43f2-98ab-71b72b85ff22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:01.132462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a766d69-c925-43f2-98ab-71b72b85ff22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:01.133633 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a766d69-c925-43f2-98ab-71b72b85ff22 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:01.142189 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1197/2392] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:01.245939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78415e23-6585-4a5b-8e95-3af71d9ea346 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:01.246551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78415e23-6585-4a5b-8e95-3af71d9ea346 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:01.246551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78415e23-6585-4a5b-8e95-3af71d9ea346 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:01.248184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1196/2393] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 21:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 21:00:01.254828 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633a231-8c04-41d7-9016-e15aa6da2b8e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:01.255253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633a231-8c04-41d7-9016-e15aa6da2b8e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:01.255744 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633a231-8c04-41d7-9016-e15aa6da2b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:01.290835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-2633a231-8c04-41d7-9016-e15aa6da2b8e None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:01.291982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2633a231-8c04-41d7-9016-e15aa6da2b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:01.378645 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1198/2394] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:01.391134 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78415e23-6585-4a5b-8e95-3af71d9ea346 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:01.393038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-78415e23-6585-4a5b-8e95-3af71d9ea346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:01.452395 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78415e23-6585-4a5b-8e95-3af71d9ea346 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:01.456049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1c89e-6460-4dd2-89ab-2c9ceb64eebe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35e7720e3bf4c9e80bdefa6a1cb8296', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:01.456733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1c89e-6460-4dd2-89ab-2c9ceb64eebe service glance] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:01.457015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1c89e-6460-4dd2-89ab-2c9ceb64eebe service glance] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:01.457405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1c89e-6460-4dd2-89ab-2c9ceb64eebe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:01.467655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1c89e-6460-4dd2-89ab-2c9ceb64eebe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:01.469098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1c89e-6460-4dd2-89ab-2c9ceb64eebe service glance] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:01.480208 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1197/2395] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:01.551007 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b090edc-a80e-426f-bc61-0083469aa6c2 None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:01.551204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b090edc-a80e-426f-bc61-0083469aa6c2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:01.551480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b090edc-a80e-426f-bc61-0083469aa6c2 None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:01.552945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1199/2396] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 21:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 21:00:01.557802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c39acc-245b-4c13-9add-5a0806e877fb None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:01.558094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c39acc-245b-4c13-9add-5a0806e877fb None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:01.558401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c39acc-245b-4c13-9add-5a0806e877fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:01.571401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:01.577719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c4c39acc-245b-4c13-9add-5a0806e877fb None None] MFA Rules not processed for user `5db827b3a66843749e04376559aa27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:01.579336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c4c39acc-245b-4c13-9add-5a0806e877fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:01.586788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1198/2397] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:01.599105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b090edc-a80e-426f-bc61-0083469aa6c2 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:01.600843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2b090edc-a80e-426f-bc61-0083469aa6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:01.614566 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b090edc-a80e-426f-bc61-0083469aa6c2 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:01.618155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c78405-dff2-4dda-8006-12bc7a5b6cfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:01.618784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c78405-dff2-4dda-8006-12bc7a5b6cfc service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:01.619071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c78405-dff2-4dda-8006-12bc7a5b6cfc service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:01.619350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c78405-dff2-4dda-8006-12bc7a5b6cfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:01.627699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c78405-dff2-4dda-8006-12bc7a5b6cfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:01.629174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c78405-dff2-4dda-8006-12bc7a5b6cfc service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:01.637490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1200/2398] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:01.719924 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60e0ce-4feb-4adb-a4a6-ee6abad180ac None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:01.720248 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60e0ce-4feb-4adb-a4a6-ee6abad180ac None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:01.720620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60e0ce-4feb-4adb-a4a6-ee6abad180ac None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:01.721940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1199/2399] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 21:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 21:00:01.727470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f97f6d-32f3-4ac8-942d-5178d48f2778 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:01.728431 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f97f6d-32f3-4ac8-942d-5178d48f2778 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:01.728745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f97f6d-32f3-4ac8-942d-5178d48f2778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:01.746470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-28f97f6d-32f3-4ac8-942d-5178d48f2778 None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:01.748604 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-28f97f6d-32f3-4ac8-942d-5178d48f2778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:01.758533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1201/2400] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:01.769342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60e0ce-4feb-4adb-a4a6-ee6abad180ac None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:01.771033 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dd60e0ce-4feb-4adb-a4a6-ee6abad180ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:01.785073 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60e0ce-4feb-4adb-a4a6-ee6abad180ac None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:01.788684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1fc4a-2c7c-440a-9273-81eb25316f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:01.789641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1fc4a-2c7c-440a-9273-81eb25316f81 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:01.789929 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1fc4a-2c7c-440a-9273-81eb25316f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:01.790839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1fc4a-2c7c-440a-9273-81eb25316f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:01.801245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1fc4a-2c7c-440a-9273-81eb25316f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:01.802609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1fc4a-2c7c-440a-9273-81eb25316f81 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:01.813309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1200/2401] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:01.879004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f97f6d-32f3-4ac8-942d-5178d48f2778 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:01.880498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-28f97f6d-32f3-4ac8-942d-5178d48f2778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:01.949700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f97f6d-32f3-4ac8-942d-5178d48f2778 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:01.955241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db1398-894e-480c-8e2f-c401d2725017 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:01.955904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db1398-894e-480c-8e2f-c401d2725017 service placement] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:01.956193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db1398-894e-480c-8e2f-c401d2725017 service placement] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:01.956515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db1398-894e-480c-8e2f-c401d2725017 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:01.958365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b5db1398-894e-480c-8e2f-c401d2725017 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:01.970773 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db1398-894e-480c-8e2f-c401d2725017 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:01.972377 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db1398-894e-480c-8e2f-c401d2725017 service placement] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:01.983907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1202/2402] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:03.765463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1fc4a-2c7c-440a-9273-81eb25316f81 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:03.774059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1fc4a-2c7c-440a-9273-81eb25316f81 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:03.778294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0975f-2bd4-42e7-b5eb-4bfea07e223f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:03.778920 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0975f-2bd4-42e7-b5eb-4bfea07e223f service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:03.779202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0975f-2bd4-42e7-b5eb-4bfea07e223f service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:03.779478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0975f-2bd4-42e7-b5eb-4bfea07e223f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:03.781391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-9ed0975f-2bd4-42e7-b5eb-4bfea07e223f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:03.790715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0975f-2bd4-42e7-b5eb-4bfea07e223f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:03.792101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0975f-2bd4-42e7-b5eb-4bfea07e223f service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:03.800004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1201/2403] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:04.685647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776ea0-8a3e-4a4e-9887-c7f435953510 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:04.686351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776ea0-8a3e-4a4e-9887-c7f435953510 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:04.686351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776ea0-8a3e-4a4e-9887-c7f435953510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:04.705697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-06776ea0-8a3e-4a4e-9887-c7f435953510 None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:04.706789 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-06776ea0-8a3e-4a4e-9887-c7f435953510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:04.734169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:04.770162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1203/2404] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 21:00:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:04.788754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0975f-2bd4-42e7-b5eb-4bfea07e223f service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:04.790694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-9ed0975f-2bd4-42e7-b5eb-4bfea07e223f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:04.876632 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0975f-2bd4-42e7-b5eb-4bfea07e223f service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:04.885009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe240a-deb7-41a5-b80b-a37328b6b092 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:04.885806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe240a-deb7-41a5-b80b-a37328b6b092 service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:04.886207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe240a-deb7-41a5-b80b-a37328b6b092 service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:04.886588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe240a-deb7-41a5-b80b-a37328b6b092 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:04.896808 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe240a-deb7-41a5-b80b-a37328b6b092 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:04.898610 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe240a-deb7-41a5-b80b-a37328b6b092 service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:04.910043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1202/2405] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 21:00:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 21:00:14.762514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=11 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:14.904772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=44 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:32.595793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06776ea0-8a3e-4a4e-9887-c7f435953510 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:32.597809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-06776ea0-8a3e-4a4e-9887-c7f435953510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:32.621615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06776ea0-8a3e-4a4e-9887-c7f435953510 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:32.628064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b1ae6-61d3-4599-b853-b70fd11eb8ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:32.628749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b1ae6-61d3-4599-b853-b70fd11eb8ba service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:32.629038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b1ae6-61d3-4599-b853-b70fd11eb8ba service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:32.629319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b1ae6-61d3-4599-b853-b70fd11eb8ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:32.631110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a15b1ae6-61d3-4599-b853-b70fd11eb8ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:32.645458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b1ae6-61d3-4599-b853-b70fd11eb8ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:32.647090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b1ae6-61d3-4599-b853-b70fd11eb8ba service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:32.661157 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1204/2406] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:00:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:00:35.847846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f4a9-ed2d-4f0d-8f9b-962e3e55e05d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:35.848398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f4a9-ed2d-4f0d-8f9b-962e3e55e05d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:35.848507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f4a9-ed2d-4f0d-8f9b-962e3e55e05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:35.868147 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-2460f4a9-ed2d-4f0d-8f9b-962e3e55e05d None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:00:35.869410 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2460f4a9-ed2d-4f0d-8f9b-962e3e55e05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:00:35.877078 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1203/2407] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jul 19 21:00:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:00:35.890065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b1ae6-61d3-4599-b853-b70fd11eb8ba service nova] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:00:35.902419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b1ae6-61d3-4599-b853-b70fd11eb8ba service nova] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:00:35.907463 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0a03a-26af-401c-a9f2-141fe423602f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:00:35.908094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0a03a-26af-401c-a9f2-141fe423602f service swift] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:00:35.908412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0a03a-26af-401c-a9f2-141fe423602f service swift] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:00:35.908692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0a03a-26af-401c-a9f2-141fe423602f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:00:35.916766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0a03a-26af-401c-a9f2-141fe423602f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:00:35.918321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0a03a-26af-401c-a9f2-141fe423602f service swift] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:00:35.926468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1205/2408] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 21:00:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 21:00:45.862615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:00:45.923201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=12 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:34.564376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9adfd-fe55-47bd-8427-6e9b677047a8 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:34.565211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9adfd-fe55-47bd-8427-6e9b677047a8 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:34.565420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9adfd-fe55-47bd-8427-6e9b677047a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:34.593070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-1fd9adfd-fe55-47bd-8427-6e9b677047a8 None None] MFA Rules not processed for user `ac1b24e50c894dcf8e887275fd91a174`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:34.594447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1fd9adfd-fe55-47bd-8427-6e9b677047a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:34.671347 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1204/2409] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:38.061383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0a03a-26af-401c-a9f2-141fe423602f service swift] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:38.085787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0a03a-26af-401c-a9f2-141fe423602f service swift] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:38.095808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f02756-9bbc-4e4e-ab1e-56b73ea11f15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:38.096603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f02756-9bbc-4e4e-ab1e-56b73ea11f15 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:38.096947 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f02756-9bbc-4e4e-ab1e-56b73ea11f15 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:38.097285 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f02756-9bbc-4e4e-ab1e-56b73ea11f15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:38.099554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-79f02756-9bbc-4e4e-ab1e-56b73ea11f15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:38.120221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f02756-9bbc-4e4e-ab1e-56b73ea11f15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:38.121906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f02756-9bbc-4e4e-ab1e-56b73ea11f15 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:38.136864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1206/2410] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:44.659663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=25 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:46.045615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9adfd-fe55-47bd-8427-6e9b677047a8 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:46.047642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1fd9adfd-fe55-47bd-8427-6e9b677047a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:46.089725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9adfd-fe55-47bd-8427-6e9b677047a8 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:46.097754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c9ddb-353a-4042-be03-fd0260a8b393 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:46.098648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c9ddb-353a-4042-be03-fd0260a8b393 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:46.099048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c9ddb-353a-4042-be03-fd0260a8b393 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:46.099327 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c9ddb-353a-4042-be03-fd0260a8b393 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:46.101569 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e10c9ddb-353a-4042-be03-fd0260a8b393 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:46.119155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c9ddb-353a-4042-be03-fd0260a8b393 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:46.121269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c9ddb-353a-4042-be03-fd0260a8b393 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:46.143123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1205/2411] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:01:48.130059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=6 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:01:51.621599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456d458-994d-402c-86c9-3dbe9b64ce44 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:51.622282 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456d458-994d-402c-86c9-3dbe9b64ce44 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:51.622704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456d458-994d-402c-86c9-3dbe9b64ce44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:51.649390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e456d458-994d-402c-86c9-3dbe9b64ce44 None None] MFA Rules not processed for user `d4c6ceff33ee449193fe6d4f7b2c01ec`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:51.650737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e456d458-994d-402c-86c9-3dbe9b64ce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:51.768553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1207/2412] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:55.089728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e608de0-0eac-4f3e-bf3b-8436fad9c01e None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:55.090713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e608de0-0eac-4f3e-bf3b-8436fad9c01e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:55.090713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e608de0-0eac-4f3e-bf3b-8436fad9c01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:55.115522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-7e608de0-0eac-4f3e-bf3b-8436fad9c01e None None] MFA Rules not processed for user `1f026e791bf24a9fab1aaedb794e6229`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:01:55.117011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7e608de0-0eac-4f3e-bf3b-8436fad9c01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:55.231320 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1206/2413] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:01:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:01:55.878174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456d458-994d-402c-86c9-3dbe9b64ce44 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:01:55.895612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456d458-994d-402c-86c9-3dbe9b64ce44 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:01:55.901900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783b239-e719-4659-bdfc-e7fc408f8ad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:01:55.902948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783b239-e719-4659-bdfc-e7fc408f8ad5 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:01:55.903403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783b239-e719-4659-bdfc-e7fc408f8ad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:01:55.903880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783b239-e719-4659-bdfc-e7fc408f8ad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:01:55.907091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d783b239-e719-4659-bdfc-e7fc408f8ad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:01:55.924353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783b239-e719-4659-bdfc-e7fc408f8ad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:01:55.927106 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783b239-e719-4659-bdfc-e7fc408f8ad5 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:01:55.942433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1208/2414] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:01:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:02:05.212781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=31 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:02:05.935136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=31 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:02:13.562331 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb681cf5-6d10-4120-9c68-c80cfc3e9854 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:13.563326 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb681cf5-6d10-4120-9c68-c80cfc3e9854 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:13.563490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb681cf5-6d10-4120-9c68-c80cfc3e9854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:13.603220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-bb681cf5-6d10-4120-9c68-c80cfc3e9854 None None] MFA Rules not processed for user `5634da4012cc4d468f84409736c28e09`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:02:13.605603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bb681cf5-6d10-4120-9c68-c80cfc3e9854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:13.743042 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1207/2415] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:02:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:02:23.738373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=37 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:02:45.407389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783b239-e719-4659-bdfc-e7fc408f8ad5 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:45.409403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d783b239-e719-4659-bdfc-e7fc408f8ad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:45.544883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783b239-e719-4659-bdfc-e7fc408f8ad5 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:45.548698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6da75-1ee5-496f-abf4-f8654b4a09f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:45.549363 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6da75-1ee5-496f-abf4-f8654b4a09f1 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:45.549653 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6da75-1ee5-496f-abf4-f8654b4a09f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:45.549935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6da75-1ee5-496f-abf4-f8654b4a09f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:45.551849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-efd6da75-1ee5-496f-abf4-f8654b4a09f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:45.564502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6da75-1ee5-496f-abf4-f8654b4a09f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:45.566058 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6da75-1ee5-496f-abf4-f8654b4a09f1 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:45.577780 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1209/2416] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:02:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:02:52.991467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb681cf5-6d10-4120-9c68-c80cfc3e9854 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:02:52.993673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bb681cf5-6d10-4120-9c68-c80cfc3e9854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:53.054695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb681cf5-6d10-4120-9c68-c80cfc3e9854 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:02:53.061770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa03456-9948-4d07-9f80-571996590be1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:02:53.062693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa03456-9948-4d07-9f80-571996590be1 service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:02:53.063088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa03456-9948-4d07-9f80-571996590be1 service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:02:53.063562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa03456-9948-4d07-9f80-571996590be1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:02:53.066241 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-faa03456-9948-4d07-9f80-571996590be1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:02:53.082018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa03456-9948-4d07-9f80-571996590be1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:02:53.083842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa03456-9948-4d07-9f80-571996590be1 service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:02:53.097254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1208/2417] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:02:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:02:55.572696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=48 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:03.091774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=18 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:06.313132 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6da75-1ee5-496f-abf4-f8654b4a09f1 service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:06.336993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6da75-1ee5-496f-abf4-f8654b4a09f1 service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:06.341113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d37138-6102-4de3-b051-410c0be3402b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:06.341786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d37138-6102-4de3-b051-410c0be3402b service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:06.342056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d37138-6102-4de3-b051-410c0be3402b service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:06.342331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d37138-6102-4de3-b051-410c0be3402b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:06.344498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-66d37138-6102-4de3-b051-410c0be3402b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:06.478675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d37138-6102-4de3-b051-410c0be3402b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:06.481174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d37138-6102-4de3-b051-410c0be3402b service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:06.507811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1210/2418] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:03:10.784955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa03456-9948-4d07-9f80-571996590be1 service placement] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:10.809452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa03456-9948-4d07-9f80-571996590be1 service placement] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:10.818519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb2035-4b3f-4862-b541-4072c9e25329 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:10.820101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb2035-4b3f-4862-b541-4072c9e25329 service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:10.821068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb2035-4b3f-4862-b541-4072c9e25329 service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:10.821891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb2035-4b3f-4862-b541-4072c9e25329 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:10.825976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-06cb2035-4b3f-4862-b541-4072c9e25329 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:10.843743 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d37138-6102-4de3-b051-410c0be3402b service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:10.847074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb2035-4b3f-4862-b541-4072c9e25329 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:10.847269 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-66d37138-6102-4de3-b051-410c0be3402b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:10.851127 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb2035-4b3f-4862-b541-4072c9e25329 service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:10.868788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1209/2419] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:10.880275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d37138-6102-4de3-b051-410c0be3402b service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:10.888660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bd4f9-4944-4ef9-92f7-30e77dff8380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:10.890958 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bd4f9-4944-4ef9-92f7-30e77dff8380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:10.891458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bd4f9-4944-4ef9-92f7-30e77dff8380 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:10.891952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bd4f9-4944-4ef9-92f7-30e77dff8380 admin admin] PATH_INFO: `/v3/users/ac1b24e50c894dcf8e887275fd91a174` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:10.894228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bd4f9-4944-4ef9-92f7-30e77dff8380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1b24e50c894dcf8e887275fd91a174)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:10.898677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bd4f9-4944-4ef9-92f7-30e77dff8380 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:10.978207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cf9bd4f9-4944-4ef9-92f7-30e77dff8380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1b24e50c894dcf8e887275fd91a174'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:10.986605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cf9bd4f9-4944-4ef9-92f7-30e77dff8380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1b24e50c894dcf8e887275fd91a174'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:10.988009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cf9bd4f9-4944-4ef9-92f7-30e77dff8380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1b24e50c894dcf8e887275fd91a174'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:11.007029 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cf9bd4f9-4944-4ef9-92f7-30e77dff8380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1b24e50c894dcf8e887275fd91a174'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:11.014907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-cf9bd4f9-4944-4ef9-92f7-30e77dff8380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1b24e50c894dcf8e887275fd91a174'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:11.021510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1211/2420] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:10 2022] DELETE /identity/v3/users/ac1b24e50c894dcf8e887275fd91a174 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:11.260491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb2035-4b3f-4862-b541-4072c9e25329 service placement] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:11.274900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb2035-4b3f-4862-b541-4072c9e25329 service placement] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:11.348447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:11.349467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:11.349886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:11.350269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] PATH_INFO: `/v3/projects/d5af714b14ba471980868293588841fd` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:11.352072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5af714b14ba471980868293588841fd)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:11.354466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:11.412118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5af714b14ba471980868293588841fd', 'request_id': 'req-4b684893-733c-460d-a891-45d2b6b8bbae'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:11.422304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5af714b14ba471980868293588841fd', 'request_id': 'req-4b684893-733c-460d-a891-45d2b6b8bbae'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:11.431656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5af714b14ba471980868293588841fd', 'request_id': 'req-4b684893-733c-460d-a891-45d2b6b8bbae'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:11.446341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] The token cache is being invalidate because project d5af714b14ba471980868293588841fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:11.446636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:11.447316 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4b684893-733c-460d-a891-45d2b6b8bbae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:11.449013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1210/2421] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:11 2022] DELETE /identity/v3/projects/d5af714b14ba471980868293588841fd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:11.460117 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30511147-e32b-46c9-8758-463ac5665e08 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:11.460548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30511147-e32b-46c9-8758-463ac5665e08 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:11.460834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30511147-e32b-46c9-8758-463ac5665e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:11.505047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:11.538619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-30511147-e32b-46c9-8758-463ac5665e08 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:11.540785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-30511147-e32b-46c9-8758-463ac5665e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:11.652235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1212/2422] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:11.660779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b086bc-0803-4579-ad1b-b0777698dde6 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:11.661066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b086bc-0803-4579-ad1b-b0777698dde6 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:11.661349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b086bc-0803-4579-ad1b-b0777698dde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:11.681588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-37b086bc-0803-4579-ad1b-b0777698dde6 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:11.682906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-37b086bc-0803-4579-ad1b-b0777698dde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:11.692249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1211/2423] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:11.697308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30511147-e32b-46c9-8758-463ac5665e08 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:11.708360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30511147-e32b-46c9-8758-463ac5665e08 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:11.714804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e751a-f39d-41f5-adc3-fc4691a19c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:11.715494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e751a-f39d-41f5-adc3-fc4691a19c93 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:11.715786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e751a-f39d-41f5-adc3-fc4691a19c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:11.716064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e751a-f39d-41f5-adc3-fc4691a19c93 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:11.716853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e751a-f39d-41f5-adc3-fc4691a19c93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:03:11.717162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e751a-f39d-41f5-adc3-fc4691a19c93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:11.718812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e751a-f39d-41f5-adc3-fc4691a19c93 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:11.732812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1213/2424] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:03:11.737701 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b086bc-0803-4579-ad1b-b0777698dde6 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:11.748277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b086bc-0803-4579-ad1b-b0777698dde6 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:11.753472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f8d3-8976-400b-b0d5-9033b22cff35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:11.754159 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f8d3-8976-400b-b0d5-9033b22cff35 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:11.754444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f8d3-8976-400b-b0d5-9033b22cff35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:11.754720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8f8d3-8976-400b-b0d5-9033b22cff35 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:11.755446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8f8d3-8976-400b-b0d5-9033b22cff35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:11.757958 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8f8d3-8976-400b-b0d5-9033b22cff35 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:11.789426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1212/2425] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:11 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:11.794022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e751a-f39d-41f5-adc3-fc4691a19c93 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:11.805539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e751a-f39d-41f5-adc3-fc4691a19c93 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:11.891556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11450665-805a-43c9-b5e6-e34782db3230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:11.892262 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11450665-805a-43c9-b5e6-e34782db3230 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:11.892615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11450665-805a-43c9-b5e6-e34782db3230 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:11.892899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11450665-805a-43c9-b5e6-e34782db3230 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:11.893629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11450665-805a-43c9-b5e6-e34782db3230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:11.895625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11450665-805a-43c9-b5e6-e34782db3230 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:11.920692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1214/2426] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:11 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:11.925821 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f8d3-8976-400b-b0d5-9033b22cff35 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:11.940501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8f8d3-8976-400b-b0d5-9033b22cff35 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:11.949371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810325af-cc8c-4283-bf25-b4318b09db10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:11.950520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810325af-cc8c-4283-bf25-b4318b09db10 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:11.951153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810325af-cc8c-4283-bf25-b4318b09db10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:11.951512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810325af-cc8c-4283-bf25-b4318b09db10 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:11.953351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810325af-cc8c-4283-bf25-b4318b09db10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:11.956404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810325af-cc8c-4283-bf25-b4318b09db10 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:11.976445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1213/2427] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:11.982557 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11450665-805a-43c9-b5e6-e34782db3230 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:11.997683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11450665-805a-43c9-b5e6-e34782db3230 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:12.002763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19755e49-5ead-4231-8a6f-0c7ce688b144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:12.003452 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19755e49-5ead-4231-8a6f-0c7ce688b144 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:12.003741 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19755e49-5ead-4231-8a6f-0c7ce688b144 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:12.004020 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19755e49-5ead-4231-8a6f-0c7ce688b144 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:12.004809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19755e49-5ead-4231-8a6f-0c7ce688b144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:12.007130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19755e49-5ead-4231-8a6f-0c7ce688b144 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:12.019832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1215/2428] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:12.026196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810325af-cc8c-4283-bf25-b4318b09db10 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:12.037736 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810325af-cc8c-4283-bf25-b4318b09db10 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:12.043018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ff6db-35e3-4778-abe0-01ec3ad5390f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:12.043847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ff6db-35e3-4778-abe0-01ec3ad5390f admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:12.044186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ff6db-35e3-4778-abe0-01ec3ad5390f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:12.044559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ff6db-35e3-4778-abe0-01ec3ad5390f admin admin] PATH_INFO: `/v3/projects/e76e331e55014bd98d8a88e1be06b038/users/47205be3ce2f452f9af44b6c079b430e/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:12.072062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ff6db-35e3-4778-abe0-01ec3ad5390f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e76e331e55014bd98d8a88e1be06b038, user_id=47205be3ce2f452f9af44b6c079b430e, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:12.077976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ff6db-35e3-4778-abe0-01ec3ad5390f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:12.099012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1214/2429] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:12 2022] PUT /identity/v3/projects/e76e331e55014bd98d8a88e1be06b038/users/47205be3ce2f452f9af44b6c079b430e/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:12.110163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbad290-8025-4efe-9898-547fcbeaab0e None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:12.110440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbad290-8025-4efe-9898-547fcbeaab0e None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:12.110728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbad290-8025-4efe-9898-547fcbeaab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:12.152949 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-cbbad290-8025-4efe-9898-547fcbeaab0e None None] MFA Rules not processed for user `47205be3ce2f452f9af44b6c079b430e`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:12.155215 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-cbbad290-8025-4efe-9898-547fcbeaab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:12.224983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:12.238242 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1216/2430] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:03:12.256622 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ff6db-35e3-4778-abe0-01ec3ad5390f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:12.259529 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fb9ff6db-35e3-4778-abe0-01ec3ad5390f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:12.295068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:12.397228 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ff6db-35e3-4778-abe0-01ec3ad5390f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:12.403436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117427b-5aaf-4f84-a057-4c93c9dcc913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:12.404130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117427b-5aaf-4f84-a057-4c93c9dcc913 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:12.404449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117427b-5aaf-4f84-a057-4c93c9dcc913 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:12.404732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117427b-5aaf-4f84-a057-4c93c9dcc913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:12.414994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117427b-5aaf-4f84-a057-4c93c9dcc913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:12.416768 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117427b-5aaf-4f84-a057-4c93c9dcc913 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:12.427683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1215/2431] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:12.676292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbad290-8025-4efe-9898-547fcbeaab0e None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:12.750923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbad290-8025-4efe-9898-547fcbeaab0e None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:12.756571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:12.757596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:12.757923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:12.758259 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] PATH_INFO: `/v3/users/47205be3ce2f452f9af44b6c079b430e` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:12.759900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] RBAC: Authorizing `identity:delete_user(user_id=47205be3ce2f452f9af44b6c079b430e)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:12.762107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:12.818605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47205be3ce2f452f9af44b6c079b430e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:12.827347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47205be3ce2f452f9af44b6c079b430e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:12.828111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47205be3ce2f452f9af44b6c079b430e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:12.842451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47205be3ce2f452f9af44b6c079b430e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:12.849500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47205be3ce2f452f9af44b6c079b430e'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:12.857260 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1217/2432] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:12 2022] DELETE /identity/v3/users/47205be3ce2f452f9af44b6c079b430e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:12.861845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa5595-217d-4747-b529-5f26b224bcbf None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:12.862374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa5595-217d-4747-b529-5f26b224bcbf None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:12.862374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa5595-217d-4747-b529-5f26b224bcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:12.880948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-73aa5595-217d-4747-b529-5f26b224bcbf None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:12.882184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-73aa5595-217d-4747-b529-5f26b224bcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:12.951342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1216/2433] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:12.962530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:12.964154 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:13.044827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e29ce2-bdd7-4bbc-9697-2235e27147de admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:13.050175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b75445-17a3-493f-b301-4706f892e83b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:13.050984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b75445-17a3-493f-b301-4706f892e83b service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:13.051271 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b75445-17a3-493f-b301-4706f892e83b service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:13.051549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b75445-17a3-493f-b301-4706f892e83b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:13.063720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b75445-17a3-493f-b301-4706f892e83b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:13.065205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b75445-17a3-493f-b301-4706f892e83b service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:13.076900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1218/2434] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:13.409147 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa5595-217d-4747-b529-5f26b224bcbf None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:13.411097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-73aa5595-217d-4747-b529-5f26b224bcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:13.426782 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa5595-217d-4747-b529-5f26b224bcbf None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:13.434564 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:13.435448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:13.435785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:13.436070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] PATH_INFO: `/v3/projects/e76e331e55014bd98d8a88e1be06b038` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:13.437424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e76e331e55014bd98d8a88e1be06b038)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:13.440598 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:13.498818 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e76e331e55014bd98d8a88e1be06b038', 'request_id': 'req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:13.509832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e76e331e55014bd98d8a88e1be06b038', 'request_id': 'req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:13.516901 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e76e331e55014bd98d8a88e1be06b038', 'request_id': 'req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:13.523418 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:13.532608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] The token cache is being invalidate because project e76e331e55014bd98d8a88e1be06b038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:13.533055 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:13.534139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c8fdd8dd-d6c0-4626-a342-ed5fd308eb3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:13.535757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1217/2435] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:13 2022] DELETE /identity/v3/projects/e76e331e55014bd98d8a88e1be06b038 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:13.541073 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c15d2-c36d-407f-a0de-0c4ec3905f63 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:13.541547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c15d2-c36d-407f-a0de-0c4ec3905f63 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:13.541940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c15d2-c36d-407f-a0de-0c4ec3905f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:13.549646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:13.565527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0e6c15d2-c36d-407f-a0de-0c4ec3905f63 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:13.567990 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0e6c15d2-c36d-407f-a0de-0c4ec3905f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:13.640534 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1219/2436] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:13.653906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cb7e9-904b-41e6-b8fd-c487decd275e None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:13.654433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cb7e9-904b-41e6-b8fd-c487decd275e None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:13.654937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cb7e9-904b-41e6-b8fd-c487decd275e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:13.677469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-9c0cb7e9-904b-41e6-b8fd-c487decd275e None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:13.679287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-9c0cb7e9-904b-41e6-b8fd-c487decd275e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:13.688812 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1218/2437] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:13.694118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c15d2-c36d-407f-a0de-0c4ec3905f63 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:13.706850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c15d2-c36d-407f-a0de-0c4ec3905f63 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:13.711821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f6a45-32ef-473d-83c5-126006a05284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:13.712545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f6a45-32ef-473d-83c5-126006a05284 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:13.712838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f6a45-32ef-473d-83c5-126006a05284 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:13.713113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f6a45-32ef-473d-83c5-126006a05284 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:13.713887 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f6a45-32ef-473d-83c5-126006a05284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:03:13.714194 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f6a45-32ef-473d-83c5-126006a05284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:13.715816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f6a45-32ef-473d-83c5-126006a05284 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:13.728921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1220/2438] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 21:03:13.733767 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cb7e9-904b-41e6-b8fd-c487decd275e None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:13.745318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cb7e9-904b-41e6-b8fd-c487decd275e None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:13.750564 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccd295-2f10-4f5f-8f7d-328c434848e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:13.751243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccd295-2f10-4f5f-8f7d-328c434848e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:13.751526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccd295-2f10-4f5f-8f7d-328c434848e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:13.751801 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bccd295-2f10-4f5f-8f7d-328c434848e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:13.752539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccd295-2f10-4f5f-8f7d-328c434848e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:13.754520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bccd295-2f10-4f5f-8f7d-328c434848e3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:13.785278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1219/2439] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:13 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:13.790147 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f6a45-32ef-473d-83c5-126006a05284 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:13.801302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f6a45-32ef-473d-83c5-126006a05284 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:13.873532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001c1d5-a929-4565-8ad3-d6bd59d19a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:13.874230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001c1d5-a929-4565-8ad3-d6bd59d19a44 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:13.874521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001c1d5-a929-4565-8ad3-d6bd59d19a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:13.874800 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001c1d5-a929-4565-8ad3-d6bd59d19a44 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:13.875533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001c1d5-a929-4565-8ad3-d6bd59d19a44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:13.877646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001c1d5-a929-4565-8ad3-d6bd59d19a44 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:13.903320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1221/2440] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:13 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:13.907989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccd295-2f10-4f5f-8f7d-328c434848e3 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:13.920465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bccd295-2f10-4f5f-8f7d-328c434848e3 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:13.926288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc5fb0-5032-4b0f-8763-57b96160fb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:13.927050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc5fb0-5032-4b0f-8763-57b96160fb79 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:13.927376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc5fb0-5032-4b0f-8763-57b96160fb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:13.927691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc5fb0-5032-4b0f-8763-57b96160fb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:13.928677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc5fb0-5032-4b0f-8763-57b96160fb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:13.930550 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc5fb0-5032-4b0f-8763-57b96160fb79 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:13.945056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1220/2441] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:13.952106 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001c1d5-a929-4565-8ad3-d6bd59d19a44 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:13.971041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001c1d5-a929-4565-8ad3-d6bd59d19a44 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:13.979409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058734a4-49d0-4f05-abfc-48e7155911aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:13.980967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058734a4-49d0-4f05-abfc-48e7155911aa admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:13.981467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058734a4-49d0-4f05-abfc-48e7155911aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:13.981953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058734a4-49d0-4f05-abfc-48e7155911aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:13.983458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058734a4-49d0-4f05-abfc-48e7155911aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:13.986681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058734a4-49d0-4f05-abfc-48e7155911aa admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:14.003671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1222/2442] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:14.009166 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc5fb0-5032-4b0f-8763-57b96160fb79 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:14.020116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc5fb0-5032-4b0f-8763-57b96160fb79 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:14.025711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9442447-a65e-4c9a-99df-9ef225bf0085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:14.026711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9442447-a65e-4c9a-99df-9ef225bf0085 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:14.027031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9442447-a65e-4c9a-99df-9ef225bf0085 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:14.027279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9442447-a65e-4c9a-99df-9ef225bf0085 admin admin] PATH_INFO: `/v3/projects/368d3de26d5a4e27bdb251d3f0b7a4cd/users/1c1d4c389d214ff9ab6e4f48385e1e4b/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:14.046982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9442447-a65e-4c9a-99df-9ef225bf0085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=368d3de26d5a4e27bdb251d3f0b7a4cd, user_id=1c1d4c389d214ff9ab6e4f48385e1e4b, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:14.049820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9442447-a65e-4c9a-99df-9ef225bf0085 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:14.066687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1221/2443] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:14 2022] PUT /identity/v3/projects/368d3de26d5a4e27bdb251d3f0b7a4cd/users/1c1d4c389d214ff9ab6e4f48385e1e4b/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:14.080367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3c695-8ce1-4a44-a85e-675b42ec19aa None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:14.080860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3c695-8ce1-4a44-a85e-675b42ec19aa None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:14.081340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3c695-8ce1-4a44-a85e-675b42ec19aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:14.118321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-67e3c695-8ce1-4a44-a85e-675b42ec19aa None None] MFA Rules not processed for user `1c1d4c389d214ff9ab6e4f48385e1e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:14.119784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-67e3c695-8ce1-4a44-a85e-675b42ec19aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:14.192164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:14.199476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1223/2444] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:14.205893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9442447-a65e-4c9a-99df-9ef225bf0085 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:14.277925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9442447-a65e-4c9a-99df-9ef225bf0085 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:14.286408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76846bb6-4fc6-4393-9f8f-0ce0d769305f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:14.287105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846bb6-4fc6-4393-9f8f-0ce0d769305f admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:14.287390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846bb6-4fc6-4393-9f8f-0ce0d769305f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:14.287664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846bb6-4fc6-4393-9f8f-0ce0d769305f admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:14.288406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76846bb6-4fc6-4393-9f8f-0ce0d769305f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:14.290373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76846bb6-4fc6-4393-9f8f-0ce0d769305f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:14.326773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1222/2445] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:14 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:03:14.331425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3c695-8ce1-4a44-a85e-675b42ec19aa None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:14.344244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3c695-8ce1-4a44-a85e-675b42ec19aa None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:14.447595 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a083df-dbc1-4d08-b963-5c1a462c8a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:14.448726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a083df-dbc1-4d08-b963-5c1a462c8a9f admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:14.449243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a083df-dbc1-4d08-b963-5c1a462c8a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:14.449758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a083df-dbc1-4d08-b963-5c1a462c8a9f admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:14.451136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a083df-dbc1-4d08-b963-5c1a462c8a9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:14.455981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a083df-dbc1-4d08-b963-5c1a462c8a9f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:14.485748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1224/2446] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:14 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:14.490569 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76846bb6-4fc6-4393-9f8f-0ce0d769305f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:14.505190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76846bb6-4fc6-4393-9f8f-0ce0d769305f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:14.510531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ece5c-728e-423e-bed6-905dccb198c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:14.511229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ece5c-728e-423e-bed6-905dccb198c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:14.511567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ece5c-728e-423e-bed6-905dccb198c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:14.511863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ece5c-728e-423e-bed6-905dccb198c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:14.512681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ece5c-728e-423e-bed6-905dccb198c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:14.514416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ece5c-728e-423e-bed6-905dccb198c5 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:14.527692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1223/2447] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:14.533252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a083df-dbc1-4d08-b963-5c1a462c8a9f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:14.545605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a083df-dbc1-4d08-b963-5c1a462c8a9f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:14.550589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e02506-7d39-4d72-b9d9-8243a25d53f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:14.551544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e02506-7d39-4d72-b9d9-8243a25d53f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:14.551845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e02506-7d39-4d72-b9d9-8243a25d53f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:14.552093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e02506-7d39-4d72-b9d9-8243a25d53f5 admin admin] PATH_INFO: `/v3/projects/7fdfc9d5a0ec4d4dbafa471ce44bea61/users/07f18ae349214e1db59de4aed0f13668/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:14.569951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e02506-7d39-4d72-b9d9-8243a25d53f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fdfc9d5a0ec4d4dbafa471ce44bea61, user_id=07f18ae349214e1db59de4aed0f13668, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:14.573135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e02506-7d39-4d72-b9d9-8243a25d53f5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:14.589629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1225/2448] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:14 2022] PUT /identity/v3/projects/7fdfc9d5a0ec4d4dbafa471ce44bea61/users/07f18ae349214e1db59de4aed0f13668/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:14.594172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ece5c-728e-423e-bed6-905dccb198c5 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:14.606193 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ece5c-728e-423e-bed6-905dccb198c5 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:14.683804 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6666b-add0-4ceb-98bd-fc3cf3b7aea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:14.684994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6666b-add0-4ceb-98bd-fc3cf3b7aea7 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:14.685481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6666b-add0-4ceb-98bd-fc3cf3b7aea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:14.685931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6666b-add0-4ceb-98bd-fc3cf3b7aea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:14.686951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6666b-add0-4ceb-98bd-fc3cf3b7aea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:14.689761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6666b-add0-4ceb-98bd-fc3cf3b7aea7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:14.711160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1224/2449] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:14.715583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e02506-7d39-4d72-b9d9-8243a25d53f5 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:14.727634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e02506-7d39-4d72-b9d9-8243a25d53f5 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:14.733148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615e0f6-509d-4418-96e9-243faf8b7139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:14.734061 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615e0f6-509d-4418-96e9-243faf8b7139 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:14.734345 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615e0f6-509d-4418-96e9-243faf8b7139 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:14.734622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615e0f6-509d-4418-96e9-243faf8b7139 admin admin] PATH_INFO: `/v3/projects/7fdfc9d5a0ec4d4dbafa471ce44bea61/users/07f18ae349214e1db59de4aed0f13668/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:14.736856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615e0f6-509d-4418-96e9-243faf8b7139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fdfc9d5a0ec4d4dbafa471ce44bea61, user_id=07f18ae349214e1db59de4aed0f13668, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:14.739526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615e0f6-509d-4418-96e9-243faf8b7139 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:14.753783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1226/2450] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:14 2022] PUT /identity/v3/projects/7fdfc9d5a0ec4d4dbafa471ce44bea61/users/07f18ae349214e1db59de4aed0f13668/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:14.765021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3214d6-b245-4ae4-8aa8-d59521187301 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:14.765326 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3214d6-b245-4ae4-8aa8-d59521187301 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:14.765577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3214d6-b245-4ae4-8aa8-d59521187301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:14.771802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:14.797912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-1c3214d6-b245-4ae4-8aa8-d59521187301 None None] MFA Rules not processed for user `07f18ae349214e1db59de4aed0f13668`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:14.799191 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1c3214d6-b245-4ae4-8aa8-d59521187301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:14.879238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1225/2451] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:03:14.893063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615e0f6-509d-4418-96e9-243faf8b7139 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:14.894933 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a615e0f6-509d-4418-96e9-243faf8b7139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:14.965694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615e0f6-509d-4418-96e9-243faf8b7139 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:14.969473 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c46ade-e333-42ef-bf44-07950ac27dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:14.970144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c46ade-e333-42ef-bf44-07950ac27dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:14.970425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c46ade-e333-42ef-bf44-07950ac27dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:14.970715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c46ade-e333-42ef-bf44-07950ac27dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:14.982440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c46ade-e333-42ef-bf44-07950ac27dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:14.983894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c46ade-e333-42ef-bf44-07950ac27dc3 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:14.996165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1227/2452] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:15.430413 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3214d6-b245-4ae4-8aa8-d59521187301 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:15.505683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3214d6-b245-4ae4-8aa8-d59521187301 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:15.510823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:15.511713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:15.511997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:15.512280 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] PATH_INFO: `/v3/users/1c1d4c389d214ff9ab6e4f48385e1e4b` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:15.513551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1d4c389d214ff9ab6e4f48385e1e4b)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:15.515496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:15.570099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1d4c389d214ff9ab6e4f48385e1e4b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:15.579512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1d4c389d214ff9ab6e4f48385e1e4b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:15.588576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1d4c389d214ff9ab6e4f48385e1e4b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:15.589373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1d4c389d214ff9ab6e4f48385e1e4b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:15.602133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1d4c389d214ff9ab6e4f48385e1e4b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:15.608629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1226/2453] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:15 2022] DELETE /identity/v3/users/1c1d4c389d214ff9ab6e4f48385e1e4b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:15.613739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df192c-d12a-4ca9-a37d-bd1df6a57dec None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:15.614098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df192c-d12a-4ca9-a37d-bd1df6a57dec None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:15.614416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df192c-d12a-4ca9-a37d-bd1df6a57dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:15.635409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-f9df192c-d12a-4ca9-a37d-bd1df6a57dec None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:15.636964 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f9df192c-d12a-4ca9-a37d-bd1df6a57dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:15.699251 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:15.713895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1228/2454] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:15.731221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:15.733643 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:15.751511 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:15.844308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1be5c-251e-4df4-aad9-61a9eb34fe6c admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:15.850979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f58bf4-f8a4-4272-b566-d7b32b2f2447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:15.852116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f58bf4-f8a4-4272-b566-d7b32b2f2447 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:15.852682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f58bf4-f8a4-4272-b566-d7b32b2f2447 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:15.853201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f58bf4-f8a4-4272-b566-d7b32b2f2447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:15.871962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f58bf4-f8a4-4272-b566-d7b32b2f2447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:15.874885 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f58bf4-f8a4-4272-b566-d7b32b2f2447 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:15.894617 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1227/2455] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:16.250181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df192c-d12a-4ca9-a37d-bd1df6a57dec None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:16.252704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f9df192c-d12a-4ca9-a37d-bd1df6a57dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:16.276667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df192c-d12a-4ca9-a37d-bd1df6a57dec None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:16.284829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:16.286413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:16.286930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:16.287498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] PATH_INFO: `/v3/projects/368d3de26d5a4e27bdb251d3f0b7a4cd` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:16.289874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=368d3de26d5a4e27bdb251d3f0b7a4cd)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:16.293808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:16.359289 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '368d3de26d5a4e27bdb251d3f0b7a4cd', 'request_id': 'req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:16.370930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '368d3de26d5a4e27bdb251d3f0b7a4cd', 'request_id': 'req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:16.379521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '368d3de26d5a4e27bdb251d3f0b7a4cd', 'request_id': 'req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:16.390583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] The token cache is being invalidate because project 368d3de26d5a4e27bdb251d3f0b7a4cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:16.390872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:16.391543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:16.393139 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1229/2456] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:16 2022] DELETE /identity/v3/projects/368d3de26d5a4e27bdb251d3f0b7a4cd => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:16.397498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f58bf4-f8a4-4272-b566-d7b32b2f2447 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:16.408509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f58bf4-f8a4-4272-b566-d7b32b2f2447 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:16.482182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55395d9b-ea85-407d-907f-f02248ef4aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:16.483354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55395d9b-ea85-407d-907f-f02248ef4aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:16.483753 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55395d9b-ea85-407d-907f-f02248ef4aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:16.484072 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55395d9b-ea85-407d-907f-f02248ef4aa0 admin admin] PATH_INFO: `/v3/users/07f18ae349214e1db59de4aed0f13668` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:16.485654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55395d9b-ea85-407d-907f-f02248ef4aa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f18ae349214e1db59de4aed0f13668)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:16.488227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55395d9b-ea85-407d-907f-f02248ef4aa0 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:16.546387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-55395d9b-ea85-407d-907f-f02248ef4aa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f18ae349214e1db59de4aed0f13668'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:16.557069 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-55395d9b-ea85-407d-907f-f02248ef4aa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f18ae349214e1db59de4aed0f13668'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:16.566424 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-55395d9b-ea85-407d-907f-f02248ef4aa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f18ae349214e1db59de4aed0f13668'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:16.567453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-55395d9b-ea85-407d-907f-f02248ef4aa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f18ae349214e1db59de4aed0f13668'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:16.583526 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-55395d9b-ea85-407d-907f-f02248ef4aa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f18ae349214e1db59de4aed0f13668'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:16.590919 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1228/2457] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:16 2022] DELETE /identity/v3/users/07f18ae349214e1db59de4aed0f13668 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:16.924977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:16.927095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:16.950397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef209f-aae1-4a95-ae5a-bfec6ef160d4 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:17.027897 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:17.028835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:17.029129 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:17.029412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] PATH_INFO: `/v3/projects/7fdfc9d5a0ec4d4dbafa471ce44bea61` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:17.030791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fdfc9d5a0ec4d4dbafa471ce44bea61)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:17.032792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:17.090838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fdfc9d5a0ec4d4dbafa471ce44bea61', 'request_id': 'req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:17.099590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fdfc9d5a0ec4d4dbafa471ce44bea61', 'request_id': 'req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:17.108094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fdfc9d5a0ec4d4dbafa471ce44bea61', 'request_id': 'req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:17.120489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] The token cache is being invalidate because project 7fdfc9d5a0ec4d4dbafa471ce44bea61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:17.120912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:17.121688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ae3f18b5-f981-4d43-84c3-d7c63b10afa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:17.123485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1230/2458] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:16 2022] DELETE /identity/v3/projects/7fdfc9d5a0ec4d4dbafa471ce44bea61 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:17.130059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e30fc-11e6-4466-a3b6-d72e17000e35 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:17.130591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e30fc-11e6-4466-a3b6-d72e17000e35 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:17.131581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e30fc-11e6-4466-a3b6-d72e17000e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:17.154431 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e67e30fc-11e6-4466-a3b6-d72e17000e35 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:17.155730 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e67e30fc-11e6-4466-a3b6-d72e17000e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:17.184534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:17.248406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1229/2459] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:17.262790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fb67c-04d7-4f94-9504-5108f00e5dec None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:17.263828 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fb67c-04d7-4f94-9504-5108f00e5dec None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:17.264186 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fb67c-04d7-4f94-9504-5108f00e5dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:17.288441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-d05fb67c-04d7-4f94-9504-5108f00e5dec None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:17.289772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d05fb67c-04d7-4f94-9504-5108f00e5dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:17.298885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1231/2460] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:17.304362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e30fc-11e6-4466-a3b6-d72e17000e35 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:17.315706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e30fc-11e6-4466-a3b6-d72e17000e35 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:17.321593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc77a9-7518-437d-ba22-04c2150acf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:17.322282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc77a9-7518-437d-ba22-04c2150acf0e admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:17.322569 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc77a9-7518-437d-ba22-04c2150acf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:17.322852 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc77a9-7518-437d-ba22-04c2150acf0e admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:17.323604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc77a9-7518-437d-ba22-04c2150acf0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:03:17.323958 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc77a9-7518-437d-ba22-04c2150acf0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:17.325664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc77a9-7518-437d-ba22-04c2150acf0e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:17.338686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1230/2461] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 21:03:17.343584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fb67c-04d7-4f94-9504-5108f00e5dec None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:17.350200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:17.360944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fb67c-04d7-4f94-9504-5108f00e5dec None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:17.370225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfef5e-75e7-4773-959a-f13055334ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:17.371022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfef5e-75e7-4773-959a-f13055334ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:17.371229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfef5e-75e7-4773-959a-f13055334ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:17.371512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfef5e-75e7-4773-959a-f13055334ef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:17.372241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfef5e-75e7-4773-959a-f13055334ef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:17.374245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfef5e-75e7-4773-959a-f13055334ef6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:17.404320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1232/2462] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:17 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:17.409338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc77a9-7518-437d-ba22-04c2150acf0e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:17.424518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc77a9-7518-437d-ba22-04c2150acf0e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:17.501883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4627d04-d6aa-47bb-81a6-f77bddcdb089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:17.502950 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4627d04-d6aa-47bb-81a6-f77bddcdb089 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:17.503369 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4627d04-d6aa-47bb-81a6-f77bddcdb089 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:17.503741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4627d04-d6aa-47bb-81a6-f77bddcdb089 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:17.504717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4627d04-d6aa-47bb-81a6-f77bddcdb089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:17.506955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4627d04-d6aa-47bb-81a6-f77bddcdb089 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:17.533024 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1231/2463] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:17 2022] POST /identity/v3/users => generated 283 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:17.537760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfef5e-75e7-4773-959a-f13055334ef6 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:17.549827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfef5e-75e7-4773-959a-f13055334ef6 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:17.555405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923f6e9-b919-459d-a819-080adeb090c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:17.556090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923f6e9-b919-459d-a819-080adeb090c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:17.556802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923f6e9-b919-459d-a819-080adeb090c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:17.557175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923f6e9-b919-459d-a819-080adeb090c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:17.557944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923f6e9-b919-459d-a819-080adeb090c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:17.559602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923f6e9-b919-459d-a819-080adeb090c9 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:17.571659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1233/2464] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:17.576845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4627d04-d6aa-47bb-81a6-f77bddcdb089 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:17.593182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4627d04-d6aa-47bb-81a6-f77bddcdb089 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:17.599671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c13539-19e2-4bbc-b958-253bbe703d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:17.600575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c13539-19e2-4bbc-b958-253bbe703d22 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:17.600904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c13539-19e2-4bbc-b958-253bbe703d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:17.601270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c13539-19e2-4bbc-b958-253bbe703d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:17.602092 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c13539-19e2-4bbc-b958-253bbe703d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:17.603846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c13539-19e2-4bbc-b958-253bbe703d22 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:17.615826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1232/2465] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:17.620629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923f6e9-b919-459d-a819-080adeb090c9 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:17.632469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923f6e9-b919-459d-a819-080adeb090c9 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:17.637724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37226aec-2e27-4db1-bdf2-d1cfcd9e5c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:17.638655 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37226aec-2e27-4db1-bdf2-d1cfcd9e5c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:17.638940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37226aec-2e27-4db1-bdf2-d1cfcd9e5c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:17.639218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37226aec-2e27-4db1-bdf2-d1cfcd9e5c96 admin admin] PATH_INFO: `/v3/projects/3f1dc0a4642948338ce62fc57248608e/users/bcbac2c3fc37482aa15f50e8bfcb7434/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:17.657293 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37226aec-2e27-4db1-bdf2-d1cfcd9e5c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1dc0a4642948338ce62fc57248608e, user_id=bcbac2c3fc37482aa15f50e8bfcb7434, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:17.660012 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37226aec-2e27-4db1-bdf2-d1cfcd9e5c96 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:17.678191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1234/2466] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:17 2022] PUT /identity/v3/projects/3f1dc0a4642948338ce62fc57248608e/users/bcbac2c3fc37482aa15f50e8bfcb7434/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:17.689530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1614-4c22-47d5-a9a9-417a7df17ccb None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:17.689891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1614-4c22-47d5-a9a9-417a7df17ccb None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:17.690216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1614-4c22-47d5-a9a9-417a7df17ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:17.723256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-981d1614-4c22-47d5-a9a9-417a7df17ccb None None] MFA Rules not processed for user `bcbac2c3fc37482aa15f50e8bfcb7434`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:17.725454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-981d1614-4c22-47d5-a9a9-417a7df17ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:17.813290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1233/2467] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:17.819105 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37226aec-2e27-4db1-bdf2-d1cfcd9e5c96 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:17.901123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37226aec-2e27-4db1-bdf2-d1cfcd9e5c96 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:17.907330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9625a-9797-4386-ace4-62b2dba4b77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:17.908233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9625a-9797-4386-ace4-62b2dba4b77e admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:17.908841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9625a-9797-4386-ace4-62b2dba4b77e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:17.909059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9625a-9797-4386-ace4-62b2dba4b77e admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:17.909712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9625a-9797-4386-ace4-62b2dba4b77e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:17.912607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9625a-9797-4386-ace4-62b2dba4b77e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:17.949551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1235/2468] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:17 2022] POST /identity/v3/projects => generated 363 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:17.954877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d1614-4c22-47d5-a9a9-417a7df17ccb None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:17.967271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d1614-4c22-47d5-a9a9-417a7df17ccb None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:17.993337 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:18.040125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ddb2d-1ee3-456f-b0be-352ca9e95f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:18.040983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ddb2d-1ee3-456f-b0be-352ca9e95f04 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:18.041355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ddb2d-1ee3-456f-b0be-352ca9e95f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:18.041784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ddb2d-1ee3-456f-b0be-352ca9e95f04 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:18.042652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ddb2d-1ee3-456f-b0be-352ca9e95f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:18.045251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ddb2d-1ee3-456f-b0be-352ca9e95f04 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:18.068353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1234/2469] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:17 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:18.072880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9625a-9797-4386-ace4-62b2dba4b77e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:18.086623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9625a-9797-4386-ace4-62b2dba4b77e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:18.091871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed198e7-4cbb-436e-9b8d-5ad2115ff97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:18.092574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed198e7-4cbb-436e-9b8d-5ad2115ff97a admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:18.092859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed198e7-4cbb-436e-9b8d-5ad2115ff97a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:18.093144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed198e7-4cbb-436e-9b8d-5ad2115ff97a admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:18.093881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed198e7-4cbb-436e-9b8d-5ad2115ff97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:18.095494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed198e7-4cbb-436e-9b8d-5ad2115ff97a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:18.108250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1236/2470] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:18.113235 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ddb2d-1ee3-456f-b0be-352ca9e95f04 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:18.125336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ddb2d-1ee3-456f-b0be-352ca9e95f04 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:18.130386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01be55-e02a-41c0-bd8c-038bf1b25856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:18.131903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01be55-e02a-41c0-bd8c-038bf1b25856 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:18.132184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01be55-e02a-41c0-bd8c-038bf1b25856 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:18.132501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01be55-e02a-41c0-bd8c-038bf1b25856 admin admin] PATH_INFO: `/v3/projects/61a2210a979641d398f3d8de67362c28/users/50d1ee7771734dad96a33ca4ca51a15b/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:18.149399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01be55-e02a-41c0-bd8c-038bf1b25856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a2210a979641d398f3d8de67362c28, user_id=50d1ee7771734dad96a33ca4ca51a15b, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:18.152784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01be55-e02a-41c0-bd8c-038bf1b25856 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:18.168521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1235/2471] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:18 2022] PUT /identity/v3/projects/61a2210a979641d398f3d8de67362c28/users/50d1ee7771734dad96a33ca4ca51a15b/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:18.173196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed198e7-4cbb-436e-9b8d-5ad2115ff97a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:18.185408 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed198e7-4cbb-436e-9b8d-5ad2115ff97a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:18.253996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5310f-8ddc-4c4f-8356-056ef5ffd4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:18.254936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5310f-8ddc-4c4f-8356-056ef5ffd4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:18.255315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5310f-8ddc-4c4f-8356-056ef5ffd4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:18.255680 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5310f-8ddc-4c4f-8356-056ef5ffd4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:18.256649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5310f-8ddc-4c4f-8356-056ef5ffd4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:18.258451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5310f-8ddc-4c4f-8356-056ef5ffd4d7 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:18.270182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1237/2472] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:18.275961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01be55-e02a-41c0-bd8c-038bf1b25856 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:18.288615 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01be55-e02a-41c0-bd8c-038bf1b25856 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:18.294255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e81333-4069-4668-9012-11ee31378c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:18.295182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e81333-4069-4668-9012-11ee31378c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:18.295460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e81333-4069-4668-9012-11ee31378c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:18.295733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e81333-4069-4668-9012-11ee31378c9e admin admin] PATH_INFO: `/v3/projects/61a2210a979641d398f3d8de67362c28/users/50d1ee7771734dad96a33ca4ca51a15b/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:18.298165 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e81333-4069-4668-9012-11ee31378c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a2210a979641d398f3d8de67362c28, user_id=50d1ee7771734dad96a33ca4ca51a15b, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:18.301145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e81333-4069-4668-9012-11ee31378c9e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:18.320880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1236/2473] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:18 2022] PUT /identity/v3/projects/61a2210a979641d398f3d8de67362c28/users/50d1ee7771734dad96a33ca4ca51a15b/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:18.339068 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed7c50-cb9c-42fa-8400-1ca01fc75e3b None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:18.339386 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed7c50-cb9c-42fa-8400-1ca01fc75e3b None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:18.339673 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed7c50-cb9c-42fa-8400-1ca01fc75e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:18.370668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-02ed7c50-cb9c-42fa-8400-1ca01fc75e3b None None] MFA Rules not processed for user `50d1ee7771734dad96a33ca4ca51a15b`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:18.372360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-02ed7c50-cb9c-42fa-8400-1ca01fc75e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:18.377135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:18.484921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1238/2474] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:18 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:03:18.500362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e81333-4069-4668-9012-11ee31378c9e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:18.502171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d3e81333-4069-4668-9012-11ee31378c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:18.600718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e81333-4069-4668-9012-11ee31378c9e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:18.608823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eda3f-aae3-4a89-bcd4-0f5e53b7d1dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:18.609958 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eda3f-aae3-4a89-bcd4-0f5e53b7d1dc service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:18.610469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eda3f-aae3-4a89-bcd4-0f5e53b7d1dc service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:18.610954 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eda3f-aae3-4a89-bcd4-0f5e53b7d1dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:18.627573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519eda3f-aae3-4a89-bcd4-0f5e53b7d1dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:18.629094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519eda3f-aae3-4a89-bcd4-0f5e53b7d1dc service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:18.641135 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1237/2475] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:18 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:18.726182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed7c50-cb9c-42fa-8400-1ca01fc75e3b None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:18.736919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed7c50-cb9c-42fa-8400-1ca01fc75e3b None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:18.741190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21819d9-a284-430c-b4e1-741fa6c1899f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:18.741859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21819d9-a284-430c-b4e1-741fa6c1899f service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:18.742138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21819d9-a284-430c-b4e1-741fa6c1899f service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:18.742418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21819d9-a284-430c-b4e1-741fa6c1899f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:18.811777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21819d9-a284-430c-b4e1-741fa6c1899f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:18.813273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21819d9-a284-430c-b4e1-741fa6c1899f service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:18.825373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1239/2476] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:28.636518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=59,keystone:INSERT=5 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:28.819862 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=55 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:30.689559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eda3f-aae3-4a89-bcd4-0f5e53b7d1dc service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:30.805940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eda3f-aae3-4a89-bcd4-0f5e53b7d1dc service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:30.814690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:30.816221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:30.816765 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:30.817525 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] PATH_INFO: `/v3/users/bcbac2c3fc37482aa15f50e8bfcb7434` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:30.819790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcbac2c3fc37482aa15f50e8bfcb7434)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:30.823611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:30.889413 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcbac2c3fc37482aa15f50e8bfcb7434'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:30.898830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcbac2c3fc37482aa15f50e8bfcb7434'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:30.907612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcbac2c3fc37482aa15f50e8bfcb7434'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:30.908421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcbac2c3fc37482aa15f50e8bfcb7434'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:30.923621 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcbac2c3fc37482aa15f50e8bfcb7434'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:30.930557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1238/2477] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:30 2022] DELETE /identity/v3/users/bcbac2c3fc37482aa15f50e8bfcb7434 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:30.935893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e3398-6b82-4979-ae0c-bfaf7f53ae2c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:30.936832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e3398-6b82-4979-ae0c-bfaf7f53ae2c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:30.936832 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e3398-6b82-4979-ae0c-bfaf7f53ae2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:30.959303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-431e3398-6b82-4979-ae0c-bfaf7f53ae2c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:30.960715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-431e3398-6b82-4979-ae0c-bfaf7f53ae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:31.032835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1240/2478] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:03:31.046125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:31.047646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:31.120783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59380f8e-47c0-4971-b258-7676030df18a admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:31.124985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bd4e9-ced7-46f0-bd89-1e8a8324a998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:31.125825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bd4e9-ced7-46f0-bd89-1e8a8324a998 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:31.126218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bd4e9-ced7-46f0-bd89-1e8a8324a998 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:31.126612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bd4e9-ced7-46f0-bd89-1e8a8324a998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:31.139207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bd4e9-ced7-46f0-bd89-1e8a8324a998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:31.140875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bd4e9-ced7-46f0-bd89-1e8a8324a998 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:31.152820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1239/2479] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:31.809904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e3398-6b82-4979-ae0c-bfaf7f53ae2c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:31.811959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-431e3398-6b82-4979-ae0c-bfaf7f53ae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:31.831280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e3398-6b82-4979-ae0c-bfaf7f53ae2c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:31.840983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:31.842591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:31.843101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:31.844286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] PATH_INFO: `/v3/projects/3f1dc0a4642948338ce62fc57248608e` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:31.846874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1dc0a4642948338ce62fc57248608e)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:31.850053 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:31.920846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1dc0a4642948338ce62fc57248608e', 'request_id': 'req-ade3af67-ba82-48ff-984e-22a4fa43d18c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:31.932976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1dc0a4642948338ce62fc57248608e', 'request_id': 'req-ade3af67-ba82-48ff-984e-22a4fa43d18c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:31.941039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1dc0a4642948338ce62fc57248608e', 'request_id': 'req-ade3af67-ba82-48ff-984e-22a4fa43d18c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:31.955411 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] The token cache is being invalidate because project 3f1dc0a4642948338ce62fc57248608e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:31.955706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:31.956387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:31.957993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1241/2480] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:31 2022] DELETE /identity/v3/projects/3f1dc0a4642948338ce62fc57248608e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:31.963210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bd4e9-ced7-46f0-bd89-1e8a8324a998 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:31.974277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bd4e9-ced7-46f0-bd89-1e8a8324a998 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:32.003689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:32.060827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f58ad2-c0cf-4a23-8e62-36a05f1c887b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:32.061801 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f58ad2-c0cf-4a23-8e62-36a05f1c887b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:32.062084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f58ad2-c0cf-4a23-8e62-36a05f1c887b admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:32.062362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f58ad2-c0cf-4a23-8e62-36a05f1c887b admin admin] PATH_INFO: `/v3/users/50d1ee7771734dad96a33ca4ca51a15b` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:32.063720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f58ad2-c0cf-4a23-8e62-36a05f1c887b admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d1ee7771734dad96a33ca4ca51a15b)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:32.065791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f58ad2-c0cf-4a23-8e62-36a05f1c887b admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:32.131462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b0f58ad2-c0cf-4a23-8e62-36a05f1c887b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d1ee7771734dad96a33ca4ca51a15b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:32.145011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b0f58ad2-c0cf-4a23-8e62-36a05f1c887b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d1ee7771734dad96a33ca4ca51a15b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:32.154659 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b0f58ad2-c0cf-4a23-8e62-36a05f1c887b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d1ee7771734dad96a33ca4ca51a15b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:32.158062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b0f58ad2-c0cf-4a23-8e62-36a05f1c887b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d1ee7771734dad96a33ca4ca51a15b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:32.181930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b0f58ad2-c0cf-4a23-8e62-36a05f1c887b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d1ee7771734dad96a33ca4ca51a15b'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:32.189640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1240/2481] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:31 2022] DELETE /identity/v3/users/50d1ee7771734dad96a33ca4ca51a15b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:32.555979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:32.558123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:32.577362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3af67-ba82-48ff-984e-22a4fa43d18c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:32.663641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:32.665155 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:32.665652 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:32.666142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] PATH_INFO: `/v3/projects/61a2210a979641d398f3d8de67362c28` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:32.668531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a2210a979641d398f3d8de67362c28)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:32.672936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:32.732701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:32.764301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a2210a979641d398f3d8de67362c28', 'request_id': 'req-adf85522-6298-4343-b1b7-b6c0321112a6'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:32.775969 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a2210a979641d398f3d8de67362c28', 'request_id': 'req-adf85522-6298-4343-b1b7-b6c0321112a6'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:32.785843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a2210a979641d398f3d8de67362c28', 'request_id': 'req-adf85522-6298-4343-b1b7-b6c0321112a6'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:32.805069 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] The token cache is being invalidate because project 61a2210a979641d398f3d8de67362c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:32.805448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:32.806303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-adf85522-6298-4343-b1b7-b6c0321112a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:32.808196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1242/2482] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:32 2022] DELETE /identity/v3/projects/61a2210a979641d398f3d8de67362c28 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:32.813895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d0885-87f3-4e48-81c7-4a133b0cdc6c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:32.814330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d0885-87f3-4e48-81c7-4a133b0cdc6c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:32.814734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d0885-87f3-4e48-81c7-4a133b0cdc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:32.845586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-1c4d0885-87f3-4e48-81c7-4a133b0cdc6c None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:32.847488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1c4d0885-87f3-4e48-81c7-4a133b0cdc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:32.919007 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1241/2483] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:32.927756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad08a0-90c2-4dc7-ab18-dd123e8c23a7 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:32.928275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad08a0-90c2-4dc7-ab18-dd123e8c23a7 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:32.928669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad08a0-90c2-4dc7-ab18-dd123e8c23a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:32.950314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-55ad08a0-90c2-4dc7-ab18-dd123e8c23a7 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:32.951579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-55ad08a0-90c2-4dc7-ab18-dd123e8c23a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:32.960561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1243/2484] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:03:32.967146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d0885-87f3-4e48-81c7-4a133b0cdc6c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:32.978358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d0885-87f3-4e48-81c7-4a133b0cdc6c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:32.983878 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8a549-d0d6-4f23-8b8b-145e1c435693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:32.984577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8a549-d0d6-4f23-8b8b-145e1c435693 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:32.984864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8a549-d0d6-4f23-8b8b-145e1c435693 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:32.985146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8a549-d0d6-4f23-8b8b-145e1c435693 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:32.985902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8a549-d0d6-4f23-8b8b-145e1c435693 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:03:32.986215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8a549-d0d6-4f23-8b8b-145e1c435693 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:32.988054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8a549-d0d6-4f23-8b8b-145e1c435693 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:33.002404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1242/2485] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:03:33.009536 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad08a0-90c2-4dc7-ab18-dd123e8c23a7 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:33.027231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad08a0-90c2-4dc7-ab18-dd123e8c23a7 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:33.035208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52a68a-65bd-4907-83da-844bad685d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:33.036121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52a68a-65bd-4907-83da-844bad685d78 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:33.036649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52a68a-65bd-4907-83da-844bad685d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:33.037229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52a68a-65bd-4907-83da-844bad685d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:33.038496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52a68a-65bd-4907-83da-844bad685d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:33.042562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52a68a-65bd-4907-83da-844bad685d78 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:33.078779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1244/2486] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:33 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:33.083501 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8a549-d0d6-4f23-8b8b-145e1c435693 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:33.095219 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8a549-d0d6-4f23-8b8b-145e1c435693 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:33.127045 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:33.165595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d7001-fbaf-490e-b3a4-b655d6c3dc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:33.166353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d7001-fbaf-490e-b3a4-b655d6c3dc45 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:33.166751 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d7001-fbaf-490e-b3a4-b655d6c3dc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:33.167083 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d7001-fbaf-490e-b3a4-b655d6c3dc45 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:33.167832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d7001-fbaf-490e-b3a4-b655d6c3dc45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:33.170106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d7001-fbaf-490e-b3a4-b655d6c3dc45 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:33.197635 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1243/2487] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:33 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:33.204241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52a68a-65bd-4907-83da-844bad685d78 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:33.218505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52a68a-65bd-4907-83da-844bad685d78 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:33.224091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c299a-6230-4ee7-8bd9-41459b79c616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:33.225290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c299a-6230-4ee7-8bd9-41459b79c616 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:33.225783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c299a-6230-4ee7-8bd9-41459b79c616 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:33.226283 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c299a-6230-4ee7-8bd9-41459b79c616 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:33.227675 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c299a-6230-4ee7-8bd9-41459b79c616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:33.230877 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c299a-6230-4ee7-8bd9-41459b79c616 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:33.254757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1245/2488] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:03:33.257892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d7001-fbaf-490e-b3a4-b655d6c3dc45 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:33.276576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d7001-fbaf-490e-b3a4-b655d6c3dc45 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:33.284585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626722a-5600-4c0c-ace1-06595d062a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:33.285771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626722a-5600-4c0c-ace1-06595d062a81 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:33.286257 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626722a-5600-4c0c-ace1-06595d062a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:33.286738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626722a-5600-4c0c-ace1-06595d062a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:33.288080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626722a-5600-4c0c-ace1-06595d062a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:33.291729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626722a-5600-4c0c-ace1-06595d062a81 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:33.305033 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1244/2489] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:33.310199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c299a-6230-4ee7-8bd9-41459b79c616 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:33.322169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c299a-6230-4ee7-8bd9-41459b79c616 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:33.327357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eca4f-a243-4a68-a539-3a18c021e5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:33.328268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eca4f-a243-4a68-a539-3a18c021e5ab admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:33.328587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eca4f-a243-4a68-a539-3a18c021e5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:33.328870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eca4f-a243-4a68-a539-3a18c021e5ab admin admin] PATH_INFO: `/v3/projects/0444674c91e64ba2a7ce51e4f6d42f3e/users/086fbd3bf07f40eea5d565b173247eda/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:33.345782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431eca4f-a243-4a68-a539-3a18c021e5ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=0444674c91e64ba2a7ce51e4f6d42f3e, user_id=086fbd3bf07f40eea5d565b173247eda, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:33.348881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431eca4f-a243-4a68-a539-3a18c021e5ab admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:33.366520 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1246/2490] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:33 2022] PUT /identity/v3/projects/0444674c91e64ba2a7ce51e4f6d42f3e/users/086fbd3bf07f40eea5d565b173247eda/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:33.377502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23d541-c952-4b84-bd05-9ed715254e5a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:33.377783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23d541-c952-4b84-bd05-9ed715254e5a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:33.378056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23d541-c952-4b84-bd05-9ed715254e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:33.408411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4c23d541-c952-4b84-bd05-9ed715254e5a None None] MFA Rules not processed for user `086fbd3bf07f40eea5d565b173247eda`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:33.409662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4c23d541-c952-4b84-bd05-9ed715254e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:33.486614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1245/2491] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:03:33.491781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eca4f-a243-4a68-a539-3a18c021e5ab admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:33.589796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eca4f-a243-4a68-a539-3a18c021e5ab admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:33.595122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338c579-b291-48ff-a84d-1c1aaecb5a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:33.595959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338c579-b291-48ff-a84d-1c1aaecb5a7e admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:33.596365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338c579-b291-48ff-a84d-1c1aaecb5a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:33.596755 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338c579-b291-48ff-a84d-1c1aaecb5a7e admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:33.597569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338c579-b291-48ff-a84d-1c1aaecb5a7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:33.600646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338c579-b291-48ff-a84d-1c1aaecb5a7e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:33.632583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1247/2492] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:33 2022] POST /identity/v3/projects => generated 369 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:33.638175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23d541-c952-4b84-bd05-9ed715254e5a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:33.658434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23d541-c952-4b84-bd05-9ed715254e5a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:33.731019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c00876-e7cc-4277-b368-faa8dd274715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:33.731857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c00876-e7cc-4277-b368-faa8dd274715 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:33.732141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c00876-e7cc-4277-b368-faa8dd274715 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:33.732448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c00876-e7cc-4277-b368-faa8dd274715 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:33.733151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c00876-e7cc-4277-b368-faa8dd274715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:33.735121 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c00876-e7cc-4277-b368-faa8dd274715 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:33.759844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1246/2493] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:33 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:33.765356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338c579-b291-48ff-a84d-1c1aaecb5a7e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:33.777890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338c579-b291-48ff-a84d-1c1aaecb5a7e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:33.783006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad28d1-1de3-480f-afd9-a729a7ffcb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:33.783696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad28d1-1de3-480f-afd9-a729a7ffcb8a admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:33.783978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad28d1-1de3-480f-afd9-a729a7ffcb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:33.784256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad28d1-1de3-480f-afd9-a729a7ffcb8a admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:33.785024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad28d1-1de3-480f-afd9-a729a7ffcb8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:33.786661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad28d1-1de3-480f-afd9-a729a7ffcb8a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:33.799044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1248/2494] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:33.804027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c00876-e7cc-4277-b368-faa8dd274715 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:33.814897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c00876-e7cc-4277-b368-faa8dd274715 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:33.822772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd12206-dfd1-4868-a919-6d315fa12bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:33.823735 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd12206-dfd1-4868-a919-6d315fa12bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:33.824046 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd12206-dfd1-4868-a919-6d315fa12bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:33.824469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd12206-dfd1-4868-a919-6d315fa12bd2 admin admin] PATH_INFO: `/v3/projects/257c819ddee74e8fbd447d08434bc3b1/users/ead437fc7b1a47ebb82ea9c7e698a9b4/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:33.847730 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd12206-dfd1-4868-a919-6d315fa12bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=257c819ddee74e8fbd447d08434bc3b1, user_id=ead437fc7b1a47ebb82ea9c7e698a9b4, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:33.853470 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd12206-dfd1-4868-a919-6d315fa12bd2 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:33.874892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1247/2495] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:33 2022] PUT /identity/v3/projects/257c819ddee74e8fbd447d08434bc3b1/users/ead437fc7b1a47ebb82ea9c7e698a9b4/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:33.879790 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad28d1-1de3-480f-afd9-a729a7ffcb8a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:33.891473 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad28d1-1de3-480f-afd9-a729a7ffcb8a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:33.915879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:33.963777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21bf02-508b-4ca1-aea1-1b875adbcb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:33.964520 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21bf02-508b-4ca1-aea1-1b875adbcb42 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:33.964793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21bf02-508b-4ca1-aea1-1b875adbcb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:33.965074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21bf02-508b-4ca1-aea1-1b875adbcb42 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:33.965829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21bf02-508b-4ca1-aea1-1b875adbcb42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:33.967486 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21bf02-508b-4ca1-aea1-1b875adbcb42 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:33.979850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1249/2496] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:33.984570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd12206-dfd1-4868-a919-6d315fa12bd2 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:33.991006 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:33.999358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd12206-dfd1-4868-a919-6d315fa12bd2 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:34.005616 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fef4f7-245f-4c00-8820-c1f7289250eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:34.007124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fef4f7-245f-4c00-8820-c1f7289250eb admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:34.007453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fef4f7-245f-4c00-8820-c1f7289250eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:34.007775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fef4f7-245f-4c00-8820-c1f7289250eb admin admin] PATH_INFO: `/v3/projects/257c819ddee74e8fbd447d08434bc3b1/users/ead437fc7b1a47ebb82ea9c7e698a9b4/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:34.010366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fef4f7-245f-4c00-8820-c1f7289250eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=257c819ddee74e8fbd447d08434bc3b1, user_id=ead437fc7b1a47ebb82ea9c7e698a9b4, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:34.013352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fef4f7-245f-4c00-8820-c1f7289250eb admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:34.026111 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1248/2497] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:33 2022] PUT /identity/v3/projects/257c819ddee74e8fbd447d08434bc3b1/users/ead437fc7b1a47ebb82ea9c7e698a9b4/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:34.095951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afa8dc-95d6-4df8-bd56-1e210fc192dd None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:34.096303 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afa8dc-95d6-4df8-bd56-1e210fc192dd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:34.096733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afa8dc-95d6-4df8-bd56-1e210fc192dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:34.129983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-03afa8dc-95d6-4df8-bd56-1e210fc192dd None None] MFA Rules not processed for user `ead437fc7b1a47ebb82ea9c7e698a9b4`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:34.131458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-03afa8dc-95d6-4df8-bd56-1e210fc192dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:34.217843 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1250/2498] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:34 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:03:34.231314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fef4f7-245f-4c00-8820-c1f7289250eb admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:34.233546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-04fef4f7-245f-4c00-8820-c1f7289250eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:34.304468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fef4f7-245f-4c00-8820-c1f7289250eb admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:34.309447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e49396-8d11-4bef-b8f0-8781ed4524cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:34.310114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e49396-8d11-4bef-b8f0-8781ed4524cc service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:34.310394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e49396-8d11-4bef-b8f0-8781ed4524cc service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:34.310666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e49396-8d11-4bef-b8f0-8781ed4524cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:34.321291 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e49396-8d11-4bef-b8f0-8781ed4524cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:34.323082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e49396-8d11-4bef-b8f0-8781ed4524cc service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:34.334237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1249/2499] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:34 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:35.956752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afa8dc-95d6-4df8-bd56-1e210fc192dd None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:36.026118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afa8dc-95d6-4df8-bd56-1e210fc192dd None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:36.030696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:36.031568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:36.031851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:36.032124 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] PATH_INFO: `/v3/users/086fbd3bf07f40eea5d565b173247eda` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:36.033360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=086fbd3bf07f40eea5d565b173247eda)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:36.035284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:36.090101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '086fbd3bf07f40eea5d565b173247eda'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:36.092124 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:36.100594 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '086fbd3bf07f40eea5d565b173247eda'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:36.101443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '086fbd3bf07f40eea5d565b173247eda'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:36.115267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '086fbd3bf07f40eea5d565b173247eda'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:36.123448 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '086fbd3bf07f40eea5d565b173247eda'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:36.130133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1251/2500] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:35 2022] DELETE /identity/v3/users/086fbd3bf07f40eea5d565b173247eda => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:36.135216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f76fa-923b-4046-861f-509209eae085 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:36.135726 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f76fa-923b-4046-861f-509209eae085 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:36.135847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f76fa-923b-4046-861f-509209eae085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:36.156882 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4a8f76fa-923b-4046-861f-509209eae085 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:36.158231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4a8f76fa-923b-4046-861f-509209eae085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:36.231786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1250/2501] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:36.242824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:36.244302 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:36.319496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4eb19-16f0-4eb3-ab3a-484f274bd9cc admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:36.323567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d4d89-6e38-4b34-87d7-19ed7dc91145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:36.324223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d4d89-6e38-4b34-87d7-19ed7dc91145 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:36.324551 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d4d89-6e38-4b34-87d7-19ed7dc91145 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:36.324827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d4d89-6e38-4b34-87d7-19ed7dc91145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:36.335359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d4d89-6e38-4b34-87d7-19ed7dc91145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:36.336783 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d4d89-6e38-4b34-87d7-19ed7dc91145 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:36.348817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1252/2502] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:36.709509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f76fa-923b-4046-861f-509209eae085 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:36.712109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4a8f76fa-923b-4046-861f-509209eae085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:36.736302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f76fa-923b-4046-861f-509209eae085 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:36.744861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:36.746244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:36.746732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:36.747213 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] PATH_INFO: `/v3/projects/0444674c91e64ba2a7ce51e4f6d42f3e` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:36.749588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0444674c91e64ba2a7ce51e4f6d42f3e)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:36.754190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:36.831404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0444674c91e64ba2a7ce51e4f6d42f3e', 'request_id': 'req-5f5b546e-d671-414e-a07b-66b9dfce5e3c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:36.841579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0444674c91e64ba2a7ce51e4f6d42f3e', 'request_id': 'req-5f5b546e-d671-414e-a07b-66b9dfce5e3c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:36.848940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0444674c91e64ba2a7ce51e4f6d42f3e', 'request_id': 'req-5f5b546e-d671-414e-a07b-66b9dfce5e3c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:36.863807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] The token cache is being invalidate because project 0444674c91e64ba2a7ce51e4f6d42f3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:36.864189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:36.864917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:36.866556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1251/2503] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:36 2022] DELETE /identity/v3/projects/0444674c91e64ba2a7ce51e4f6d42f3e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:36.872737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d4d89-6e38-4b34-87d7-19ed7dc91145 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:36.890663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d4d89-6e38-4b34-87d7-19ed7dc91145 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:36.991409 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5020576-ca35-4ea3-997c-49c685848b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:36.992280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5020576-ca35-4ea3-997c-49c685848b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:36.992795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5020576-ca35-4ea3-997c-49c685848b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:36.993151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5020576-ca35-4ea3-997c-49c685848b3e admin admin] PATH_INFO: `/v3/users/ead437fc7b1a47ebb82ea9c7e698a9b4` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:36.994581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5020576-ca35-4ea3-997c-49c685848b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ead437fc7b1a47ebb82ea9c7e698a9b4)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:36.996873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5020576-ca35-4ea3-997c-49c685848b3e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:37.067482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-c5020576-ca35-4ea3-997c-49c685848b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ead437fc7b1a47ebb82ea9c7e698a9b4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:37.079138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-c5020576-ca35-4ea3-997c-49c685848b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ead437fc7b1a47ebb82ea9c7e698a9b4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:37.080203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-c5020576-ca35-4ea3-997c-49c685848b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ead437fc7b1a47ebb82ea9c7e698a9b4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:37.098575 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:37.101624 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-c5020576-ca35-4ea3-997c-49c685848b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ead437fc7b1a47ebb82ea9c7e698a9b4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:37.114246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-c5020576-ca35-4ea3-997c-49c685848b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ead437fc7b1a47ebb82ea9c7e698a9b4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:37.122359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1253/2504] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:36 2022] DELETE /identity/v3/users/ead437fc7b1a47ebb82ea9c7e698a9b4 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:37.469714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:37.471537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:37.490644 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b546e-d671-414e-a07b-66b9dfce5e3c admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:37.511598 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:37.586953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:37.587855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:37.588308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:37.588496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] PATH_INFO: `/v3/projects/257c819ddee74e8fbd447d08434bc3b1` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:37.590028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=257c819ddee74e8fbd447d08434bc3b1)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:37.591986 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:37.665405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '257c819ddee74e8fbd447d08434bc3b1', 'request_id': 'req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:37.674532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '257c819ddee74e8fbd447d08434bc3b1', 'request_id': 'req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:37.684205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '257c819ddee74e8fbd447d08434bc3b1', 'request_id': 'req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:37.697446 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] The token cache is being invalidate because project 257c819ddee74e8fbd447d08434bc3b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:37.697902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:37.698806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bbe04a56-0d68-4d9b-9937-41c5918ac1ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:37.700476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1252/2505] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:37 2022] DELETE /identity/v3/projects/257c819ddee74e8fbd447d08434bc3b1 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:37.707669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec217095-e60d-47fc-9045-d397c3fdecd1 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:37.707951 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec217095-e60d-47fc-9045-d397c3fdecd1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:37.708229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec217095-e60d-47fc-9045-d397c3fdecd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:37.728736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ec217095-e60d-47fc-9045-d397c3fdecd1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:37.730060 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ec217095-e60d-47fc-9045-d397c3fdecd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:37.809334 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1254/2506] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:37.821844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7adcd-d37e-4d00-b240-e5888df4a84d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:37.822160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7adcd-d37e-4d00-b240-e5888df4a84d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:37.822457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7adcd-d37e-4d00-b240-e5888df4a84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:37.846148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-65c7adcd-d37e-4d00-b240-e5888df4a84d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:37.847412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-65c7adcd-d37e-4d00-b240-e5888df4a84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:37.856920 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1253/2507] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:37.861711 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec217095-e60d-47fc-9045-d397c3fdecd1 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:37.872453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec217095-e60d-47fc-9045-d397c3fdecd1 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:37.877327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa91e25-d411-4a20-b535-b2187d97ace2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:37.878005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa91e25-d411-4a20-b535-b2187d97ace2 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:37.878295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa91e25-d411-4a20-b535-b2187d97ace2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:37.878577 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa91e25-d411-4a20-b535-b2187d97ace2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:37.879324 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa91e25-d411-4a20-b535-b2187d97ace2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:03:37.879637 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa91e25-d411-4a20-b535-b2187d97ace2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:37.881816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa91e25-d411-4a20-b535-b2187d97ace2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:37.895519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1255/2508] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 21:03:37.902966 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7adcd-d37e-4d00-b240-e5888df4a84d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:37.921500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7adcd-d37e-4d00-b240-e5888df4a84d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:37.929284 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3b1d1-bac4-42d1-9a18-700afee251d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:37.930279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3b1d1-bac4-42d1-9a18-700afee251d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:37.930702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3b1d1-bac4-42d1-9a18-700afee251d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:37.931112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3b1d1-bac4-42d1-9a18-700afee251d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:37.932185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3b1d1-bac4-42d1-9a18-700afee251d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:37.935602 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3b1d1-bac4-42d1-9a18-700afee251d5 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:37.978181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1254/2509] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:37 2022] POST /identity/v3/projects => generated 387 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:37.984046 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa91e25-d411-4a20-b535-b2187d97ace2 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:37.999646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa91e25-d411-4a20-b535-b2187d97ace2 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:38.084751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f846fc5-1210-4c35-bba0-180c19c5e9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:38.085815 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f846fc5-1210-4c35-bba0-180c19c5e9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.086332 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f846fc5-1210-4c35-bba0-180c19c5e9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.086825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f846fc5-1210-4c35-bba0-180c19c5e9f3 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.088123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f846fc5-1210-4c35-bba0-180c19c5e9f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:38.092391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f846fc5-1210-4c35-bba0-180c19c5e9f3 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:38.121673 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1256/2510] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:37 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:38.126315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3b1d1-bac4-42d1-9a18-700afee251d5 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:38.138331 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3b1d1-bac4-42d1-9a18-700afee251d5 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:38.143361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0502c6-6e5e-46f6-b0e3-c7d22d5761bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:38.144051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0502c6-6e5e-46f6-b0e3-c7d22d5761bc admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.144356 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0502c6-6e5e-46f6-b0e3-c7d22d5761bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.144648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0502c6-6e5e-46f6-b0e3-c7d22d5761bc admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.145408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0502c6-6e5e-46f6-b0e3-c7d22d5761bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:38.147132 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0502c6-6e5e-46f6-b0e3-c7d22d5761bc admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:38.160568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1255/2511] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:03:38.166218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f846fc5-1210-4c35-bba0-180c19c5e9f3 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:38.182680 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f846fc5-1210-4c35-bba0-180c19c5e9f3 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:38.188256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d30ab-b13b-4709-a76d-ac052b3c41c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:38.189094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d30ab-b13b-4709-a76d-ac052b3c41c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.189469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d30ab-b13b-4709-a76d-ac052b3c41c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.189793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d30ab-b13b-4709-a76d-ac052b3c41c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.190650 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d30ab-b13b-4709-a76d-ac052b3c41c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:38.192771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d30ab-b13b-4709-a76d-ac052b3c41c9 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:38.205636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1257/2512] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:38.211098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0502c6-6e5e-46f6-b0e3-c7d22d5761bc admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:38.223493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0502c6-6e5e-46f6-b0e3-c7d22d5761bc admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:38.229583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38877253-154e-4c75-90d2-1c1193f9ff6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:38.230969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38877253-154e-4c75-90d2-1c1193f9ff6f admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.231442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38877253-154e-4c75-90d2-1c1193f9ff6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.231640 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38877253-154e-4c75-90d2-1c1193f9ff6f admin admin] PATH_INFO: `/v3/projects/7ce40dc325f240658bfaf24242c8cf2c/users/5ee7ae4e346c4e23a190af871f5626bb/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.250026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38877253-154e-4c75-90d2-1c1193f9ff6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce40dc325f240658bfaf24242c8cf2c, user_id=5ee7ae4e346c4e23a190af871f5626bb, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:38.252605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38877253-154e-4c75-90d2-1c1193f9ff6f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:38.268406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1256/2513] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:38 2022] PUT /identity/v3/projects/7ce40dc325f240658bfaf24242c8cf2c/users/5ee7ae4e346c4e23a190af871f5626bb/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:38.279819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca37839-6079-4a51-b04b-2820eb84b0be None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.280102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca37839-6079-4a51-b04b-2820eb84b0be None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.280433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca37839-6079-4a51-b04b-2820eb84b0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.312041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-8ca37839-6079-4a51-b04b-2820eb84b0be None None] MFA Rules not processed for user `5ee7ae4e346c4e23a190af871f5626bb`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:38.313355 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8ca37839-6079-4a51-b04b-2820eb84b0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:38.366449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:38.405071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1258/2514] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:38.409863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38877253-154e-4c75-90d2-1c1193f9ff6f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:38.467906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:38.491174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38877253-154e-4c75-90d2-1c1193f9ff6f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:38.496723 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347075af-e61b-4e8e-a3b8-2718abfdee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:38.497425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347075af-e61b-4e8e-a3b8-2718abfdee17 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.497715 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347075af-e61b-4e8e-a3b8-2718abfdee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.497997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347075af-e61b-4e8e-a3b8-2718abfdee17 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.498759 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347075af-e61b-4e8e-a3b8-2718abfdee17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:38.501094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347075af-e61b-4e8e-a3b8-2718abfdee17 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:38.534275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1257/2515] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:38 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:03:38.539331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca37839-6079-4a51-b04b-2820eb84b0be None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:38.553457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca37839-6079-4a51-b04b-2820eb84b0be None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:38.642890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4b2a-7c39-44d8-a919-a99e9c453238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:38.643502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4b2a-7c39-44d8-a919-a99e9c453238 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.643795 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4b2a-7c39-44d8-a919-a99e9c453238 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.644079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab4b2a-7c39-44d8-a919-a99e9c453238 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.644838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab4b2a-7c39-44d8-a919-a99e9c453238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:38.647034 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab4b2a-7c39-44d8-a919-a99e9c453238 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:38.673346 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1259/2516] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:38 2022] POST /identity/v3/users => generated 300 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:38.678948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347075af-e61b-4e8e-a3b8-2718abfdee17 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:38.690463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347075af-e61b-4e8e-a3b8-2718abfdee17 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:38.695977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc0d46-4fcd-448c-b330-83a0259cfcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:38.697021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc0d46-4fcd-448c-b330-83a0259cfcd1 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.697021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc0d46-4fcd-448c-b330-83a0259cfcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.697309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc0d46-4fcd-448c-b330-83a0259cfcd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.698146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc0d46-4fcd-448c-b330-83a0259cfcd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:38.699840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc0d46-4fcd-448c-b330-83a0259cfcd1 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:38.713011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1258/2517] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:38.717652 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4b2a-7c39-44d8-a919-a99e9c453238 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:38.730757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab4b2a-7c39-44d8-a919-a99e9c453238 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:38.736479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe2c1-c873-4f64-8d76-f486994a5695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:38.737280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe2c1-c873-4f64-8d76-f486994a5695 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.737590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe2c1-c873-4f64-8d76-f486994a5695 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.737924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe2c1-c873-4f64-8d76-f486994a5695 admin admin] PATH_INFO: `/v3/projects/9d87d1fe4a0945e08b629909ce39dc4c/users/a2cb2263672e4974b47fa5993daeed51/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.756884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffe2c1-c873-4f64-8d76-f486994a5695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d87d1fe4a0945e08b629909ce39dc4c, user_id=a2cb2263672e4974b47fa5993daeed51, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:38.759940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffe2c1-c873-4f64-8d76-f486994a5695 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:38.776683 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1260/2518] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:38 2022] PUT /identity/v3/projects/9d87d1fe4a0945e08b629909ce39dc4c/users/a2cb2263672e4974b47fa5993daeed51/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:38.782708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc0d46-4fcd-448c-b330-83a0259cfcd1 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:38.795576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc0d46-4fcd-448c-b330-83a0259cfcd1 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:38.865565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e85d95-095d-4f9e-a9c0-808d4f2aee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:38.866260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e85d95-095d-4f9e-a9c0-808d4f2aee7b admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.866551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e85d95-095d-4f9e-a9c0-808d4f2aee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.866833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e85d95-095d-4f9e-a9c0-808d4f2aee7b admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.867576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e85d95-095d-4f9e-a9c0-808d4f2aee7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:38.869316 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e85d95-095d-4f9e-a9c0-808d4f2aee7b admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:38.882195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1259/2519] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:03:38.887176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe2c1-c873-4f64-8d76-f486994a5695 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:38.900015 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe2c1-c873-4f64-8d76-f486994a5695 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:38.905831 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001fcbb-3f70-4a6d-b35a-ddff3b81cae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:38.906761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001fcbb-3f70-4a6d-b35a-ddff3b81cae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.907042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001fcbb-3f70-4a6d-b35a-ddff3b81cae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.907326 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001fcbb-3f70-4a6d-b35a-ddff3b81cae3 admin admin] PATH_INFO: `/v3/projects/9d87d1fe4a0945e08b629909ce39dc4c/users/a2cb2263672e4974b47fa5993daeed51/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.909599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001fcbb-3f70-4a6d-b35a-ddff3b81cae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d87d1fe4a0945e08b629909ce39dc4c, user_id=a2cb2263672e4974b47fa5993daeed51, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:38.912713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001fcbb-3f70-4a6d-b35a-ddff3b81cae3 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:38.926333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1261/2520] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:38 2022] PUT /identity/v3/projects/9d87d1fe4a0945e08b629909ce39dc4c/users/a2cb2263672e4974b47fa5993daeed51/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:38.938772 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2577a3-e48a-40ec-91b1-70fcf56b77e3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:38.939198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2577a3-e48a-40ec-91b1-70fcf56b77e3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:38.939524 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2577a3-e48a-40ec-91b1-70fcf56b77e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:38.973049 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-2e2577a3-e48a-40ec-91b1-70fcf56b77e3 None None] MFA Rules not processed for user `a2cb2263672e4974b47fa5993daeed51`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:38.974943 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2e2577a3-e48a-40ec-91b1-70fcf56b77e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:39.059051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1260/2521] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:03:39.072983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001fcbb-3f70-4a6d-b35a-ddff3b81cae3 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:39.074896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-6001fcbb-3f70-4a6d-b35a-ddff3b81cae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:39.146814 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001fcbb-3f70-4a6d-b35a-ddff3b81cae3 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:39.151090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c48cc2-f005-4981-8e5c-cdcf29df3d1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:39.151873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48cc2-f005-4981-8e5c-cdcf29df3d1e service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:39.152270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48cc2-f005-4981-8e5c-cdcf29df3d1e service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:39.152597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48cc2-f005-4981-8e5c-cdcf29df3d1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:39.163982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c48cc2-f005-4981-8e5c-cdcf29df3d1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:39.165496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c48cc2-f005-4981-8e5c-cdcf29df3d1e service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:39.177082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1262/2522] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:03:39.851636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2577a3-e48a-40ec-91b1-70fcf56b77e3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:39.925854 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:39.969249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2577a3-e48a-40ec-91b1-70fcf56b77e3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:39.977444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:39.979078 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:39.979599 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:39.980114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] PATH_INFO: `/v3/users/5ee7ae4e346c4e23a190af871f5626bb` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:39.982603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ee7ae4e346c4e23a190af871f5626bb)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:39.986856 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:40.070577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ee7ae4e346c4e23a190af871f5626bb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:40.080378 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ee7ae4e346c4e23a190af871f5626bb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:40.089315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ee7ae4e346c4e23a190af871f5626bb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:40.090248 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ee7ae4e346c4e23a190af871f5626bb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:40.105861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ee7ae4e346c4e23a190af871f5626bb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:40.113340 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1261/2523] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:39 2022] DELETE /identity/v3/users/5ee7ae4e346c4e23a190af871f5626bb => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:40.118263 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ff1cc-fdf0-49ba-bfc5-46cd6be084ff None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:40.119027 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ff1cc-fdf0-49ba-bfc5-46cd6be084ff None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:40.119027 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ff1cc-fdf0-49ba-bfc5-46cd6be084ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:40.140467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-734ff1cc-fdf0-49ba-bfc5-46cd6be084ff None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:40.141787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-734ff1cc-fdf0-49ba-bfc5-46cd6be084ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:40.183010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:40.221376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1263/2524] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:03:40.232463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:40.234107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:40.304504 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a418d2-227b-4e6b-a5b0-8d9c85bbfeb7 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:40.308798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731819e3-9dbf-4853-bbe2-1eb7a3c186c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:40.309611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731819e3-9dbf-4853-bbe2-1eb7a3c186c7 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:40.309942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731819e3-9dbf-4853-bbe2-1eb7a3c186c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:40.310276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731819e3-9dbf-4853-bbe2-1eb7a3c186c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:40.322141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731819e3-9dbf-4853-bbe2-1eb7a3c186c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:40.323667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731819e3-9dbf-4853-bbe2-1eb7a3c186c7 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:40.334800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1262/2525] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:40.733721 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ff1cc-fdf0-49ba-bfc5-46cd6be084ff None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:40.735431 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-734ff1cc-fdf0-49ba-bfc5-46cd6be084ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:40.751035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ff1cc-fdf0-49ba-bfc5-46cd6be084ff None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:40.757016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:40.758023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:40.758402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:40.758794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] PATH_INFO: `/v3/projects/7ce40dc325f240658bfaf24242c8cf2c` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:40.761151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ce40dc325f240658bfaf24242c8cf2c)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:40.763125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:40.847677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ce40dc325f240658bfaf24242c8cf2c', 'request_id': 'req-4df44400-4f87-422b-89bd-7c2372e0c0c5'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:40.861829 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ce40dc325f240658bfaf24242c8cf2c', 'request_id': 'req-4df44400-4f87-422b-89bd-7c2372e0c0c5'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:40.875491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ce40dc325f240658bfaf24242c8cf2c', 'request_id': 'req-4df44400-4f87-422b-89bd-7c2372e0c0c5'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:40.888946 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] The token cache is being invalidate because project 7ce40dc325f240658bfaf24242c8cf2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:40.889312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:40.890143 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:40.892337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1264/2526] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:40 2022] DELETE /identity/v3/projects/7ce40dc325f240658bfaf24242c8cf2c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:40.897437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731819e3-9dbf-4853-bbe2-1eb7a3c186c7 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:40.908683 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731819e3-9dbf-4853-bbe2-1eb7a3c186c7 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:40.976769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41048101-12bf-44df-820f-3a76a73f0143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:40.977708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41048101-12bf-44df-820f-3a76a73f0143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:40.978077 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41048101-12bf-44df-820f-3a76a73f0143 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:40.978463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41048101-12bf-44df-820f-3a76a73f0143 admin admin] PATH_INFO: `/v3/users/a2cb2263672e4974b47fa5993daeed51` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:40.979924 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41048101-12bf-44df-820f-3a76a73f0143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2cb2263672e4974b47fa5993daeed51)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:40.982032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41048101-12bf-44df-820f-3a76a73f0143 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:40.984023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:41.048899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-41048101-12bf-44df-820f-3a76a73f0143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2cb2263672e4974b47fa5993daeed51'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:41.059757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-41048101-12bf-44df-820f-3a76a73f0143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2cb2263672e4974b47fa5993daeed51'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:41.070280 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-41048101-12bf-44df-820f-3a76a73f0143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2cb2263672e4974b47fa5993daeed51'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:41.071167 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-41048101-12bf-44df-820f-3a76a73f0143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2cb2263672e4974b47fa5993daeed51'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:41.094037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-41048101-12bf-44df-820f-3a76a73f0143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2cb2263672e4974b47fa5993daeed51'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:41.105675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1263/2527] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:40 2022] DELETE /identity/v3/users/a2cb2263672e4974b47fa5993daeed51 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:41.512433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:41.517545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:41.543969 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df44400-4f87-422b-89bd-7c2372e0c0c5 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:41.644548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:41.646138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:41.646763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:41.647091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] PATH_INFO: `/v3/projects/9d87d1fe4a0945e08b629909ce39dc4c` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:41.649818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d87d1fe4a0945e08b629909ce39dc4c)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:41.654067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:41.761489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d87d1fe4a0945e08b629909ce39dc4c', 'request_id': 'req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:41.785851 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d87d1fe4a0945e08b629909ce39dc4c', 'request_id': 'req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:41.806680 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d87d1fe4a0945e08b629909ce39dc4c', 'request_id': 'req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:41.811873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:41.852794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] The token cache is being invalidate because project 9d87d1fe4a0945e08b629909ce39dc4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:41.853219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:41.854158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0965ec1a-f2ad-450f-ae43-6c8165bc5eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:41.858614 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1265/2528] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:41 2022] DELETE /identity/v3/projects/9d87d1fe4a0945e08b629909ce39dc4c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:41.867628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe485074-4df6-4f28-a61b-a709586e9f7f None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:41.868521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe485074-4df6-4f28-a61b-a709586e9f7f None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:41.869388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe485074-4df6-4f28-a61b-a709586e9f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:41.911296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-fe485074-4df6-4f28-a61b-a709586e9f7f None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:41.914357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fe485074-4df6-4f28-a61b-a709586e9f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:42.100098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1264/2529] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:42.105425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98f1ad-79ce-4800-af7e-999dabf6119d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:42.105912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98f1ad-79ce-4800-af7e-999dabf6119d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:42.106333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98f1ad-79ce-4800-af7e-999dabf6119d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:42.139909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-3d98f1ad-79ce-4800-af7e-999dabf6119d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:42.141860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3d98f1ad-79ce-4800-af7e-999dabf6119d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:42.152378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1266/2530] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:42.158690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe485074-4df6-4f28-a61b-a709586e9f7f None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:42.189519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe485074-4df6-4f28-a61b-a709586e9f7f None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:42.197015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca01bd-a028-4bab-a04a-70c6762138a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:42.198051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca01bd-a028-4bab-a04a-70c6762138a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:42.198408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca01bd-a028-4bab-a04a-70c6762138a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:42.198704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca01bd-a028-4bab-a04a-70c6762138a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:42.199693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca01bd-a028-4bab-a04a-70c6762138a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:03:42.200244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca01bd-a028-4bab-a04a-70c6762138a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:42.207070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca01bd-a028-4bab-a04a-70c6762138a4 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:42.225625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1265/2531] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 21:03:42.231819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98f1ad-79ce-4800-af7e-999dabf6119d None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:42.247070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98f1ad-79ce-4800-af7e-999dabf6119d None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:42.254555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29a06a-7fde-4e26-b8ac-b91951112d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:42.255290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29a06a-7fde-4e26-b8ac-b91951112d0c admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:42.256193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29a06a-7fde-4e26-b8ac-b91951112d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:42.256521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29a06a-7fde-4e26-b8ac-b91951112d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:42.257323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29a06a-7fde-4e26-b8ac-b91951112d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:42.259159 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29a06a-7fde-4e26-b8ac-b91951112d0c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:42.301408 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1267/2532] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:42 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:03:42.306681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca01bd-a028-4bab-a04a-70c6762138a4 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:42.320569 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca01bd-a028-4bab-a04a-70c6762138a4 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:42.413027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef836f30-c0e3-4749-8ee9-ad3e8227a5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:42.414190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836f30-c0e3-4749-8ee9-ad3e8227a5e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:42.414685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836f30-c0e3-4749-8ee9-ad3e8227a5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:42.415301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836f30-c0e3-4749-8ee9-ad3e8227a5e9 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:42.416666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef836f30-c0e3-4749-8ee9-ad3e8227a5e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:42.420705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef836f30-c0e3-4749-8ee9-ad3e8227a5e9 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:42.436953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:42.461814 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1266/2533] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:42 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:03:42.466636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29a06a-7fde-4e26-b8ac-b91951112d0c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:42.480812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29a06a-7fde-4e26-b8ac-b91951112d0c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:42.489188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8bf64-9f63-4356-805f-4098e945a479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:42.490418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8bf64-9f63-4356-805f-4098e945a479 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:42.490932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8bf64-9f63-4356-805f-4098e945a479 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:42.491438 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8bf64-9f63-4356-805f-4098e945a479 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:42.492905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8bf64-9f63-4356-805f-4098e945a479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:42.496133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8bf64-9f63-4356-805f-4098e945a479 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:42.517514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1268/2534] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:03:42.524440 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef836f30-c0e3-4749-8ee9-ad3e8227a5e9 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:42.537561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef836f30-c0e3-4749-8ee9-ad3e8227a5e9 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:42.543115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ae476-33f5-42b8-ba10-1b59daabdee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:42.543859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae476-33f5-42b8-ba10-1b59daabdee3 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:42.544182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae476-33f5-42b8-ba10-1b59daabdee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:42.544499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae476-33f5-42b8-ba10-1b59daabdee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:42.545352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ae476-33f5-42b8-ba10-1b59daabdee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:42.546994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ae476-33f5-42b8-ba10-1b59daabdee3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:42.558816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1267/2535] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:42.564256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8bf64-9f63-4356-805f-4098e945a479 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:42.576529 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8bf64-9f63-4356-805f-4098e945a479 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:42.581818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055640df-ee67-4d77-b389-c03e69985389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:42.582666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055640df-ee67-4d77-b389-c03e69985389 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:42.582948 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055640df-ee67-4d77-b389-c03e69985389 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:42.583225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055640df-ee67-4d77-b389-c03e69985389 admin admin] PATH_INFO: `/v3/projects/f7e7108b8bb84ac3a4a80fce7490e761/users/c59789d10ce843ad91fbfa639d4af70b/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:42.601856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055640df-ee67-4d77-b389-c03e69985389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e7108b8bb84ac3a4a80fce7490e761, user_id=c59789d10ce843ad91fbfa639d4af70b, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:42.604619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055640df-ee67-4d77-b389-c03e69985389 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:42.624625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1269/2536] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:42 2022] PUT /identity/v3/projects/f7e7108b8bb84ac3a4a80fce7490e761/users/c59789d10ce843ad91fbfa639d4af70b/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:42.637969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a900549-b6b6-4adc-b711-ac1d3c72cbe8 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:42.638254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a900549-b6b6-4adc-b711-ac1d3c72cbe8 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:42.638519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a900549-b6b6-4adc-b711-ac1d3c72cbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:42.684687 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-3a900549-b6b6-4adc-b711-ac1d3c72cbe8 None None] MFA Rules not processed for user `c59789d10ce843ad91fbfa639d4af70b`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:42.686871 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3a900549-b6b6-4adc-b711-ac1d3c72cbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:42.774608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1268/2537] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:42.779670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055640df-ee67-4d77-b389-c03e69985389 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:42.886497 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055640df-ee67-4d77-b389-c03e69985389 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:42.894184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2d5f0-deaa-4e96-bace-33693a79430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:42.896280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2d5f0-deaa-4e96-bace-33693a79430c admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:42.896698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2d5f0-deaa-4e96-bace-33693a79430c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:42.897130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2d5f0-deaa-4e96-bace-33693a79430c admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:42.898036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2d5f0-deaa-4e96-bace-33693a79430c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:42.900502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2d5f0-deaa-4e96-bace-33693a79430c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:42.932124 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1270/2538] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:42 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:03:42.937286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a900549-b6b6-4adc-b711-ac1d3c72cbe8 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:42.954001 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a900549-b6b6-4adc-b711-ac1d3c72cbe8 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:43.035035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f147793-fbf6-4949-944c-b8aad41cfa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:43.035635 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f147793-fbf6-4949-944c-b8aad41cfa1d admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.035956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f147793-fbf6-4949-944c-b8aad41cfa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.036239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f147793-fbf6-4949-944c-b8aad41cfa1d admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:43.037106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f147793-fbf6-4949-944c-b8aad41cfa1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:43.039173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f147793-fbf6-4949-944c-b8aad41cfa1d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:43.068620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1269/2539] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:42 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:03:43.075018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2d5f0-deaa-4e96-bace-33693a79430c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:43.091632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2d5f0-deaa-4e96-bace-33693a79430c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:43.097286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39779ef7-1fb3-4e50-b87a-de04125f77c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:43.098328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39779ef7-1fb3-4e50-b87a-de04125f77c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.098793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39779ef7-1fb3-4e50-b87a-de04125f77c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.099791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39779ef7-1fb3-4e50-b87a-de04125f77c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:43.100910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39779ef7-1fb3-4e50-b87a-de04125f77c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:43.103066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39779ef7-1fb3-4e50-b87a-de04125f77c0 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:43.122954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1271/2540] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:43.127431 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f147793-fbf6-4949-944c-b8aad41cfa1d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:43.141132 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f147793-fbf6-4949-944c-b8aad41cfa1d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:43.147355 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9a064-57e3-443c-84ca-251128bd409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:43.148146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9a064-57e3-443c-84ca-251128bd409d admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.148538 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9a064-57e3-443c-84ca-251128bd409d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.148928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9a064-57e3-443c-84ca-251128bd409d admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:43.149691 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9a064-57e3-443c-84ca-251128bd409d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:43.151987 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9a064-57e3-443c-84ca-251128bd409d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:43.165019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1270/2541] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:43.169623 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39779ef7-1fb3-4e50-b87a-de04125f77c0 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:43.181265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39779ef7-1fb3-4e50-b87a-de04125f77c0 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:43.187061 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ac890-2285-41f3-a0fa-e4e7210d1e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:43.188032 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ac890-2285-41f3-a0fa-e4e7210d1e5a admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.188392 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ac890-2285-41f3-a0fa-e4e7210d1e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.188731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ac890-2285-41f3-a0fa-e4e7210d1e5a admin admin] PATH_INFO: `/v3/projects/544e2fefbdd44c4f8effa1670cff2d0a/users/2f95af890ae1439695d268d268c66a20/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:43.207361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ac890-2285-41f3-a0fa-e4e7210d1e5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=544e2fefbdd44c4f8effa1670cff2d0a, user_id=2f95af890ae1439695d268d268c66a20, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:43.210231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ac890-2285-41f3-a0fa-e4e7210d1e5a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:43.227287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1272/2542] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:43 2022] PUT /identity/v3/projects/544e2fefbdd44c4f8effa1670cff2d0a/users/2f95af890ae1439695d268d268c66a20/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:43.238510 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07622da9-6e36-4c9f-8a24-3ddb5bf6f74c None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.238850 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07622da9-6e36-4c9f-8a24-3ddb5bf6f74c None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.239169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07622da9-6e36-4c9f-8a24-3ddb5bf6f74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:43.273124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-07622da9-6e36-4c9f-8a24-3ddb5bf6f74c None None] MFA Rules not processed for user `2f95af890ae1439695d268d268c66a20`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:43.274395 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-07622da9-6e36-4c9f-8a24-3ddb5bf6f74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:43.332508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:43.370438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1271/2543] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:03:43.375436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ac890-2285-41f3-a0fa-e4e7210d1e5a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:43.431726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:43.467906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ac890-2285-41f3-a0fa-e4e7210d1e5a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:43.473024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084e68b-ac93-41cb-a9c8-8bcabfcfa8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:43.473717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084e68b-ac93-41cb-a9c8-8bcabfcfa8cf admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.474521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084e68b-ac93-41cb-a9c8-8bcabfcfa8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.474811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084e68b-ac93-41cb-a9c8-8bcabfcfa8cf admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:43.475528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084e68b-ac93-41cb-a9c8-8bcabfcfa8cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:43.477541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084e68b-ac93-41cb-a9c8-8bcabfcfa8cf admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:43.507297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1273/2544] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:03:43 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:03:43.513490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07622da9-6e36-4c9f-8a24-3ddb5bf6f74c None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:43.526327 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07622da9-6e36-4c9f-8a24-3ddb5bf6f74c None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:43.595342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64b2fc-76c3-4c69-a60f-653f43b30a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:43.596038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64b2fc-76c3-4c69-a60f-653f43b30a54 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.596396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64b2fc-76c3-4c69-a60f-653f43b30a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.596748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64b2fc-76c3-4c69-a60f-653f43b30a54 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:43.597515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64b2fc-76c3-4c69-a60f-653f43b30a54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:43.599724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64b2fc-76c3-4c69-a60f-653f43b30a54 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:43.624279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1272/2545] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:03:43 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:03:43.630454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084e68b-ac93-41cb-a9c8-8bcabfcfa8cf admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:43.649839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084e68b-ac93-41cb-a9c8-8bcabfcfa8cf admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:43.658113 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842899e-8153-462d-8f21-60d845e0c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:43.659225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842899e-8153-462d-8f21-60d845e0c473 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.659713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842899e-8153-462d-8f21-60d845e0c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.660185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842899e-8153-462d-8f21-60d845e0c473 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:43.661577 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842899e-8153-462d-8f21-60d845e0c473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:43.665270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b842899e-8153-462d-8f21-60d845e0c473 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:43.686682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1274/2546] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:43.691259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64b2fc-76c3-4c69-a60f-653f43b30a54 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:43.705471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64b2fc-76c3-4c69-a60f-653f43b30a54 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:43.714652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b1f4f-bbe8-4b88-9f38-8c24b2360210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:43.716319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b1f4f-bbe8-4b88-9f38-8c24b2360210 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.716889 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b1f4f-bbe8-4b88-9f38-8c24b2360210 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.717394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b1f4f-bbe8-4b88-9f38-8c24b2360210 admin admin] PATH_INFO: `/v3/projects/48e8850e6e6347f88bb8b79a75470121/users/8d30e9212e104e71b5d01f10ceeaa460/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:43.741210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b1f4f-bbe8-4b88-9f38-8c24b2360210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48e8850e6e6347f88bb8b79a75470121, user_id=8d30e9212e104e71b5d01f10ceeaa460, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:43.743936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b1f4f-bbe8-4b88-9f38-8c24b2360210 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:43.769756 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1273/2547] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:43 2022] PUT /identity/v3/projects/48e8850e6e6347f88bb8b79a75470121/users/8d30e9212e104e71b5d01f10ceeaa460/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:43.777745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842899e-8153-462d-8f21-60d845e0c473 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:43.796281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b842899e-8153-462d-8f21-60d845e0c473 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:43.887772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac2e59-b526-4bcd-900e-1bbc97b37389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:43.888988 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac2e59-b526-4bcd-900e-1bbc97b37389 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.889512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac2e59-b526-4bcd-900e-1bbc97b37389 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.890019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac2e59-b526-4bcd-900e-1bbc97b37389 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:43.891400 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac2e59-b526-4bcd-900e-1bbc97b37389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:43.894480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac2e59-b526-4bcd-900e-1bbc97b37389 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:43.921923 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1275/2548] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:03:43.927402 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b1f4f-bbe8-4b88-9f38-8c24b2360210 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:43.945754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b1f4f-bbe8-4b88-9f38-8c24b2360210 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:43.953940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13503e4-f7a4-40b2-a618-c0e9aee9fc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:43.955539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13503e4-f7a4-40b2-a618-c0e9aee9fc7b admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.956023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13503e4-f7a4-40b2-a618-c0e9aee9fc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.956585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13503e4-f7a4-40b2-a618-c0e9aee9fc7b admin admin] PATH_INFO: `/v3/projects/48e8850e6e6347f88bb8b79a75470121/users/8d30e9212e104e71b5d01f10ceeaa460/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:43.960532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13503e4-f7a4-40b2-a618-c0e9aee9fc7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=48e8850e6e6347f88bb8b79a75470121, user_id=8d30e9212e104e71b5d01f10ceeaa460, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:43.965853 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13503e4-f7a4-40b2-a618-c0e9aee9fc7b admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:43.986125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1274/2549] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:03:43 2022] PUT /identity/v3/projects/48e8850e6e6347f88bb8b79a75470121/users/8d30e9212e104e71b5d01f10ceeaa460/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:43.997735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768cdf-ce95-42db-af3c-140dec807ba7 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:43.998158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768cdf-ce95-42db-af3c-140dec807ba7 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:43.998533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d768cdf-ce95-42db-af3c-140dec807ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:44.029636 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0d768cdf-ce95-42db-af3c-140dec807ba7 None None] MFA Rules not processed for user `8d30e9212e104e71b5d01f10ceeaa460`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:44.031024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0d768cdf-ce95-42db-af3c-140dec807ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:44.131072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1276/2550] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:03:44.143764 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13503e4-f7a4-40b2-a618-c0e9aee9fc7b admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:44.145761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b13503e4-f7a4-40b2-a618-c0e9aee9fc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:44.229436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13503e4-f7a4-40b2-a618-c0e9aee9fc7b admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:44.234496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0270668-d93b-4b1d-930a-4afbbbee0591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:44.235656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0270668-d93b-4b1d-930a-4afbbbee0591 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:44.236100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0270668-d93b-4b1d-930a-4afbbbee0591 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:44.236584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0270668-d93b-4b1d-930a-4afbbbee0591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:44.307507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:44.323137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0270668-d93b-4b1d-930a-4afbbbee0591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:44.325173 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0270668-d93b-4b1d-930a-4afbbbee0591 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:44.336846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1275/2551] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:46.088008 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d768cdf-ce95-42db-af3c-140dec807ba7 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:46.108162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d768cdf-ce95-42db-af3c-140dec807ba7 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:46.115018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f3a42-252b-40e6-9875-4c28d1010bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:46.116183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f3a42-252b-40e6-9875-4c28d1010bdd service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:46.116541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f3a42-252b-40e6-9875-4c28d1010bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:46.116837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f3a42-252b-40e6-9875-4c28d1010bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:46.163351 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:46.202230 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f3a42-252b-40e6-9875-4c28d1010bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:46.204410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f3a42-252b-40e6-9875-4c28d1010bdd service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:46.228423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1277/2552] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:47.018182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0270668-d93b-4b1d-930a-4afbbbee0591 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:47.020096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c0270668-d93b-4b1d-930a-4afbbbee0591 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:47.033021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0270668-d93b-4b1d-930a-4afbbbee0591 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:47.037454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77442dfd-9cdb-42a8-b660-5d783c43f82b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:47.038266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442dfd-9cdb-42a8-b660-5d783c43f82b service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:47.038567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442dfd-9cdb-42a8-b660-5d783c43f82b service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:47.038851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442dfd-9cdb-42a8-b660-5d783c43f82b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:47.049107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77442dfd-9cdb-42a8-b660-5d783c43f82b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:47.050574 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77442dfd-9cdb-42a8-b660-5d783c43f82b service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:47.063205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1276/2553] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:51.073982 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f3a42-252b-40e6-9875-4c28d1010bdd service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:51.168103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f3a42-252b-40e6-9875-4c28d1010bdd service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:51.175617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:51.176569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:51.176854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:51.177168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] PATH_INFO: `/v3/users/c59789d10ce843ad91fbfa639d4af70b` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:51.178658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] RBAC: Authorizing `identity:delete_user(user_id=c59789d10ce843ad91fbfa639d4af70b)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:51.180694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:51.254112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c59789d10ce843ad91fbfa639d4af70b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:51.264658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c59789d10ce843ad91fbfa639d4af70b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:51.265634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c59789d10ce843ad91fbfa639d4af70b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:51.287495 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c59789d10ce843ad91fbfa639d4af70b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:51.295482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c59789d10ce843ad91fbfa639d4af70b'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:51.305255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1278/2554] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:51 2022] DELETE /identity/v3/users/c59789d10ce843ad91fbfa639d4af70b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:51.312761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaea539-7656-4d81-8422-08b7af2188b8 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:51.313581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaea539-7656-4d81-8422-08b7af2188b8 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:51.313581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaea539-7656-4d81-8422-08b7af2188b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:51.335562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-bdaea539-7656-4d81-8422-08b7af2188b8 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:03:51.336860 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bdaea539-7656-4d81-8422-08b7af2188b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:51.408288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1277/2555] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:03:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:03:51.419124 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:51.420785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:51.515648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36e722-d0dd-47eb-a128-5f758fd904be admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:51.520270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98800d66-52dc-405b-ac78-1e6298023717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:51.521059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98800d66-52dc-405b-ac78-1e6298023717 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:51.521367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98800d66-52dc-405b-ac78-1e6298023717 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:51.521651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98800d66-52dc-405b-ac78-1e6298023717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:51.532140 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:51.535676 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98800d66-52dc-405b-ac78-1e6298023717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:51.538115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98800d66-52dc-405b-ac78-1e6298023717 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:51.553962 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1279/2556] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:03:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:03:51.972647 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaea539-7656-4d81-8422-08b7af2188b8 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:51.974105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bdaea539-7656-4d81-8422-08b7af2188b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:52.003094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaea539-7656-4d81-8422-08b7af2188b8 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:52.013264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:52.014893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:52.015373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:52.015859 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] PATH_INFO: `/v3/projects/f7e7108b8bb84ac3a4a80fce7490e761` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:52.018959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e7108b8bb84ac3a4a80fce7490e761)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:52.022761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:52.107172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e7108b8bb84ac3a4a80fce7490e761', 'request_id': 'req-e048d998-8aca-435f-992a-40d4bdd8bef4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:52.118904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e7108b8bb84ac3a4a80fce7490e761', 'request_id': 'req-e048d998-8aca-435f-992a-40d4bdd8bef4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:52.130737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e7108b8bb84ac3a4a80fce7490e761', 'request_id': 'req-e048d998-8aca-435f-992a-40d4bdd8bef4'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:52.150259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] The token cache is being invalidate because project f7e7108b8bb84ac3a4a80fce7490e761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:52.150694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:52.151883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:52.154788 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1278/2557] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:51 2022] DELETE /identity/v3/projects/f7e7108b8bb84ac3a4a80fce7490e761 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:52.161029 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98800d66-52dc-405b-ac78-1e6298023717 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:52.177245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98800d66-52dc-405b-ac78-1e6298023717 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:52.285785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:52.287429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:52.287910 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:52.288562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] PATH_INFO: `/v3/users/2f95af890ae1439695d268d268c66a20` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:52.290175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f95af890ae1439695d268d268c66a20)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:52.292148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:52.361127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f95af890ae1439695d268d268c66a20'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:52.372895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f95af890ae1439695d268d268c66a20'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:52.373558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f95af890ae1439695d268d268c66a20'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:52.396472 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f95af890ae1439695d268d268c66a20'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:52.410343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f95af890ae1439695d268d268c66a20'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:52.422589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1280/2558] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:52 2022] DELETE /identity/v3/users/2f95af890ae1439695d268d268c66a20 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:52.798773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:52.801458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:52.831606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048d998-8aca-435f-992a-40d4bdd8bef4 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:52.923815 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:52.924756 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:52.925038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:52.925317 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] PATH_INFO: `/v3/projects/544e2fefbdd44c4f8effa1670cff2d0a` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:52.926765 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=544e2fefbdd44c4f8effa1670cff2d0a)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:52.929171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:52.933244 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:53.126266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '544e2fefbdd44c4f8effa1670cff2d0a', 'request_id': 'req-c9179353-a1af-405a-aa57-20b55c00dfcf'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:53.135681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '544e2fefbdd44c4f8effa1670cff2d0a', 'request_id': 'req-c9179353-a1af-405a-aa57-20b55c00dfcf'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:53.143409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '544e2fefbdd44c4f8effa1670cff2d0a', 'request_id': 'req-c9179353-a1af-405a-aa57-20b55c00dfcf'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:53.156415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] The token cache is being invalidate because project 544e2fefbdd44c4f8effa1670cff2d0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:53.156727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:53.157403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:53.160040 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1279/2559] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:52 2022] DELETE /identity/v3/projects/544e2fefbdd44c4f8effa1670cff2d0a => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:03:53.164640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:53.166204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:53.180169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872c675-17a6-4955-9ba9-a19eae78e801 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:53.251701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9d1a1-fdbb-49d5-831c-8d8a092c2799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:53.252674 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9d1a1-fdbb-49d5-831c-8d8a092c2799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:53.252956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9d1a1-fdbb-49d5-831c-8d8a092c2799 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:53.253240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9d1a1-fdbb-49d5-831c-8d8a092c2799 admin admin] PATH_INFO: `/v3/users/8d30e9212e104e71b5d01f10ceeaa460` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:53.254500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9d1a1-fdbb-49d5-831c-8d8a092c2799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d30e9212e104e71b5d01f10ceeaa460)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:53.257093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9d1a1-fdbb-49d5-831c-8d8a092c2799 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:53.280556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:03:53.333002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-62e9d1a1-fdbb-49d5-831c-8d8a092c2799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d30e9212e104e71b5d01f10ceeaa460'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:53.343692 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-62e9d1a1-fdbb-49d5-831c-8d8a092c2799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d30e9212e104e71b5d01f10ceeaa460'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:53.345297 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-62e9d1a1-fdbb-49d5-831c-8d8a092c2799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d30e9212e104e71b5d01f10ceeaa460'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:53.373956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-62e9d1a1-fdbb-49d5-831c-8d8a092c2799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d30e9212e104e71b5d01f10ceeaa460'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:53.378029 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-62e9d1a1-fdbb-49d5-831c-8d8a092c2799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d30e9212e104e71b5d01f10ceeaa460'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:53.390311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1281/2560] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:03:53 2022] DELETE /identity/v3/users/8d30e9212e104e71b5d01f10ceeaa460 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:03:53.877603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:03:53.881172 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:03:53.900582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9179353-a1af-405a-aa57-20b55c00dfcf admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:03:53.982740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:03:53.985798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:03:53.986088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:03:53.987242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] PATH_INFO: `/v3/projects/48e8850e6e6347f88bb8b79a75470121` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:03:53.990502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] RBAC: Authorizing `identity:delete_project(project_id=48e8850e6e6347f88bb8b79a75470121)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:03:53.993284 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:03:54.091119 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48e8850e6e6347f88bb8b79a75470121', 'request_id': 'req-f3268bf4-db64-43d8-b9ef-24e67113f54e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:54.106108 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48e8850e6e6347f88bb8b79a75470121', 'request_id': 'req-f3268bf4-db64-43d8-b9ef-24e67113f54e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:54.125496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48e8850e6e6347f88bb8b79a75470121', 'request_id': 'req-f3268bf4-db64-43d8-b9ef-24e67113f54e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:54.141570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] The token cache is being invalidate because project 48e8850e6e6347f88bb8b79a75470121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:03:54.141845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:54.142502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-f3268bf4-db64-43d8-b9ef-24e67113f54e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:03:54.144290 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1280/2561] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:03:53 2022] DELETE /identity/v3/projects/48e8850e6e6347f88bb8b79a75470121 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:04:03.313454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d18e-ad6b-4c19-82a8-e46db0786dec None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:03.314198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d18e-ad6b-4c19-82a8-e46db0786dec None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:03.314198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d18e-ad6b-4c19-82a8-e46db0786dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:03.336631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-7f08d18e-ad6b-4c19-82a8-e46db0786dec None None] MFA Rules not processed for user `d4c6ceff33ee449193fe6d4f7b2c01ec`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:03.337913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7f08d18e-ad6b-4c19-82a8-e46db0786dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:03.417070 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1282/2562] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:04:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:04.138129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=75,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:04.633445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77f4e0-aeba-49c0-baf7-c2e6a20c75ba None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:04.633963 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77f4e0-aeba-49c0-baf7-c2e6a20c75ba None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:04.634468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77f4e0-aeba-49c0-baf7-c2e6a20c75ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:04.686246 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-0c77f4e0-aeba-49c0-baf7-c2e6a20c75ba None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:04.688618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0c77f4e0-aeba-49c0-baf7-c2e6a20c75ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:04.816896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1281/2563] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 21:04:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:04.890998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d18e-ad6b-4c19-82a8-e46db0786dec None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:04.893556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7f08d18e-ad6b-4c19-82a8-e46db0786dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:05.004472 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d18e-ad6b-4c19-82a8-e46db0786dec None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:05.010213 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bfbf77-879b-4786-81da-f862160c87d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:05.010894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfbf77-879b-4786-81da-f862160c87d5 service swift] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:05.011185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfbf77-879b-4786-81da-f862160c87d5 service swift] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:05.011471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfbf77-879b-4786-81da-f862160c87d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:05.022552 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bfbf77-879b-4786-81da-f862160c87d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:05.025798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bfbf77-879b-4786-81da-f862160c87d5 service swift] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:05.042955 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1283/2564] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 21:04:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 21:04:14.799825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=29 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:15.036023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=78,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:21.400357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77f4e0-aeba-49c0-baf7-c2e6a20c75ba None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:21.402567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0c77f4e0-aeba-49c0-baf7-c2e6a20c75ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:21.471419 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77f4e0-aeba-49c0-baf7-c2e6a20c75ba None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:21.475796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49495439-ef3a-4750-a150-0893189b9377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:21.476506 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49495439-ef3a-4750-a150-0893189b9377 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:21.476796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49495439-ef3a-4750-a150-0893189b9377 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:21.477081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49495439-ef3a-4750-a150-0893189b9377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:21.479023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-49495439-ef3a-4750-a150-0893189b9377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:21.562533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49495439-ef3a-4750-a150-0893189b9377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:21.564023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49495439-ef3a-4750-a150-0893189b9377 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:21.581656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1282/2565] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:04:21.813313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bfbf77-879b-4786-81da-f862160c87d5 service swift] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:21.814816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c9bfbf77-879b-4786-81da-f862160c87d5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:21.907142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bfbf77-879b-4786-81da-f862160c87d5 service swift] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:21.913527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37511f1-a197-495e-a4a4-ce6d22c80c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:21.914666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37511f1-a197-495e-a4a4-ce6d22c80c10 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:21.915179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37511f1-a197-495e-a4a4-ce6d22c80c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:21.915641 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37511f1-a197-495e-a4a4-ce6d22c80c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:21.918531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a37511f1-a197-495e-a4a4-ce6d22c80c10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:21.941134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37511f1-a197-495e-a4a4-ce6d22c80c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:21.943152 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37511f1-a197-495e-a4a4-ce6d22c80c10 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:21.956506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1284/2566] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:04:22.983502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213e966-a67b-41ac-95d2-a6322273266d None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:22.984401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213e966-a67b-41ac-95d2-a6322273266d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:22.984401 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213e966-a67b-41ac-95d2-a6322273266d None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:22.985516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1283/2567] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 21:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 21:04:22.993063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c11ce3-2d6b-4687-9220-6291e963183c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:22.994188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c11ce3-2d6b-4687-9220-6291e963183c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:22.994188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c11ce3-2d6b-4687-9220-6291e963183c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:23.053724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-f7c11ce3-2d6b-4687-9220-6291e963183c None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:23.055594 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f7c11ce3-2d6b-4687-9220-6291e963183c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:23.068672 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1285/2568] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 21:04:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:04:23.081781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213e966-a67b-41ac-95d2-a6322273266d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:23.083652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2213e966-a67b-41ac-95d2-a6322273266d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:23.176595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213e966-a67b-41ac-95d2-a6322273266d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:23.183903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cb912-c4de-4b32-a827-a8724d0cc43e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35e7720e3bf4c9e80bdefa6a1cb8296', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:23.183903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb912-c4de-4b32-a827-a8724d0cc43e service glance] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:23.183903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb912-c4de-4b32-a827-a8724d0cc43e service glance] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:23.183903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cb912-c4de-4b32-a827-a8724d0cc43e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:23.197565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cb912-c4de-4b32-a827-a8724d0cc43e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:23.199050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cb912-c4de-4b32-a827-a8724d0cc43e service glance] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:23.216799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1284/2569] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:04:23.321056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17099383-8159-462c-b1af-7c50797748dd None None] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:23.321573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17099383-8159-462c-b1af-7c50797748dd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:23.322053 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17099383-8159-462c-b1af-7c50797748dd None None] PATH_INFO: `/` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:23.323719 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1286/2570] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 21:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 21:04:23.330861 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5443-973c-4cdb-8d33-b2bb635ed8ca None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:23.331171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5443-973c-4cdb-8d33-b2bb635ed8ca None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:23.331417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c5443-973c-4cdb-8d33-b2bb635ed8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:23.356487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-bd1c5443-973c-4cdb-8d33-b2bb635ed8ca None None] MFA Rules not processed for user `5db827b3a66843749e04376559aa27c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:23.358711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bd1c5443-973c-4cdb-8d33-b2bb635ed8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:23.371084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1285/2571] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:23.385436 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17099383-8159-462c-b1af-7c50797748dd None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:23.398350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17099383-8159-462c-b1af-7c50797748dd None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:23.402725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e88f54-5dba-48d5-866a-f093a59b7614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:23.403472 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e88f54-5dba-48d5-866a-f093a59b7614 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:23.403769 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e88f54-5dba-48d5-866a-f093a59b7614 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:23.404044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e88f54-5dba-48d5-866a-f093a59b7614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:23.413243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e88f54-5dba-48d5-866a-f093a59b7614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:23.414817 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e88f54-5dba-48d5-866a-f093a59b7614 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:23.424779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1287/2572] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:04:23.532315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd247ce9-3e9a-47e3-b721-d8e648c130b3 None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:23.533052 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd247ce9-3e9a-47e3-b721-d8e648c130b3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:23.533555 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd247ce9-3e9a-47e3-b721-d8e648c130b3 None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:23.536462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1286/2573] 10.222.0.85 () {54 vars in 921 bytes} [Tue Jul 19 21:04:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 21:04:23.546883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9dbc8-aa56-423a-aa48-de512ee1c35f None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:23.547413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9dbc8-aa56-423a-aa48-de512ee1c35f None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:23.548290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9dbc8-aa56-423a-aa48-de512ee1c35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:23.583924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-9de9dbc8-aa56-423a-aa48-de512ee1c35f None None] MFA Rules not processed for user `928b8b2b510b4c939b37b2c0dfa57060`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:23.586109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9de9dbc8-aa56-423a-aa48-de512ee1c35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:23.607580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1288/2574] 10.222.0.85 () {58 vars in 1018 bytes} [Tue Jul 19 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:23.622221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd247ce9-3e9a-47e3-b721-d8e648c130b3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:23.625264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fd247ce9-3e9a-47e3-b721-d8e648c130b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:23.648916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd247ce9-3e9a-47e3-b721-d8e648c130b3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:23.653636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085033d5-c44e-4d01-8400-0433895eae45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:23.654421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085033d5-c44e-4d01-8400-0433895eae45 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:23.654831 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085033d5-c44e-4d01-8400-0433895eae45 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:23.655251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085033d5-c44e-4d01-8400-0433895eae45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:23.669428 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085033d5-c44e-4d01-8400-0433895eae45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:23.671038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085033d5-c44e-4d01-8400-0433895eae45 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:23.684177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1287/2575] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:04:26.287638 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc68e8-c201-4db0-a5ce-4008c3652d2c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:26.288308 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc68e8-c201-4db0-a5ce-4008c3652d2c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:26.288666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc68e8-c201-4db0-a5ce-4008c3652d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:26.318375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e8fc68e8-c201-4db0-a5ce-4008c3652d2c None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:26.319725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e8fc68e8-c201-4db0-a5ce-4008c3652d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:26.327839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1289/2576] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:26.341272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085033d5-c44e-4d01-8400-0433895eae45 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:26.354779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085033d5-c44e-4d01-8400-0433895eae45 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:26.362053 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa866a9-b4e9-48c0-9118-b707a53f138c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:26.362749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa866a9-b4e9-48c0-9118-b707a53f138c service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:26.363041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa866a9-b4e9-48c0-9118-b707a53f138c service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:26.363343 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa866a9-b4e9-48c0-9118-b707a53f138c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:26.374433 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa866a9-b4e9-48c0-9118-b707a53f138c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:26.376348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa866a9-b4e9-48c0-9118-b707a53f138c service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:26.386793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1288/2577] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 21:04:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 21:04:36.310405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=47 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:36.383089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=95 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:36.574516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc68e8-c201-4db0-a5ce-4008c3652d2c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:36.576076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e8fc68e8-c201-4db0-a5ce-4008c3652d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:36.589734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc68e8-c201-4db0-a5ce-4008c3652d2c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:36.593749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a171b2-a8ca-4c4d-aafc-4d98057e92fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:36.594426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a171b2-a8ca-4c4d-aafc-4d98057e92fd service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:36.594714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a171b2-a8ca-4c4d-aafc-4d98057e92fd service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:36.594997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a171b2-a8ca-4c4d-aafc-4d98057e92fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:36.597702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d1a171b2-a8ca-4c4d-aafc-4d98057e92fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:36.670172 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a171b2-a8ca-4c4d-aafc-4d98057e92fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:36.671634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a171b2-a8ca-4c4d-aafc-4d98057e92fd service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:36.683704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1290/2578] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:04:38.243957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa866a9-b4e9-48c0-9118-b707a53f138c service swift] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:38.245712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cfa866a9-b4e9-48c0-9118-b707a53f138c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:38.263461 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa866a9-b4e9-48c0-9118-b707a53f138c service swift] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:38.269454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325e1a4-2c84-4559-bbbc-ee2699a9d0a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:38.270128 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325e1a4-2c84-4559-bbbc-ee2699a9d0a8 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:38.270416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325e1a4-2c84-4559-bbbc-ee2699a9d0a8 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:38.270694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325e1a4-2c84-4559-bbbc-ee2699a9d0a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:38.272854 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d325e1a4-2c84-4559-bbbc-ee2699a9d0a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:38.374087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325e1a4-2c84-4559-bbbc-ee2699a9d0a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:38.375731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325e1a4-2c84-4559-bbbc-ee2699a9d0a8 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:38.399123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1289/2579] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:04:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:04:46.677842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=28 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:48.395031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=41 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:04:48.987443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a7ece-1317-4f25-8a3c-dbdc5cce7b58 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:48.988092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a7ece-1317-4f25-8a3c-dbdc5cce7b58 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:48.988092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a7ece-1317-4f25-8a3c-dbdc5cce7b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:49.010575 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-521a7ece-1317-4f25-8a3c-dbdc5cce7b58 None None] MFA Rules not processed for user `5634da4012cc4d468f84409736c28e09`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:49.012731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-521a7ece-1317-4f25-8a3c-dbdc5cce7b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:49.024845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1291/2580] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:50.222295 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688392-2e5e-4305-a8ec-ab833628186d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:50.223215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688392-2e5e-4305-a8ec-ab833628186d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:50.223215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b688392-2e5e-4305-a8ec-ab833628186d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:50.253887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-8b688392-2e5e-4305-a8ec-ab833628186d None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:04:50.255229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-8b688392-2e5e-4305-a8ec-ab833628186d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:04:50.263822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1290/2581] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jul 19 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:04:50.281875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a7ece-1317-4f25-8a3c-dbdc5cce7b58 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:04:50.298098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a7ece-1317-4f25-8a3c-dbdc5cce7b58 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:04:50.304056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334485d9-040f-4e0e-8edd-1ef4537ef695 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:04:50.304787 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334485d9-040f-4e0e-8edd-1ef4537ef695 service swift] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:04:50.305071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334485d9-040f-4e0e-8edd-1ef4537ef695 service swift] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:04:50.305359 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334485d9-040f-4e0e-8edd-1ef4537ef695 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:04:50.315330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334485d9-040f-4e0e-8edd-1ef4537ef695 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:04:50.317096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334485d9-040f-4e0e-8edd-1ef4537ef695 service swift] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:04:50.327827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1292/2582] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 21:04:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 21:05:00.248307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:05:00.323055 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=9 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:05:06.647568 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b688392-2e5e-4305-a8ec-ab833628186d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:06.658538 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b688392-2e5e-4305-a8ec-ab833628186d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:06.662156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42275acb-6d47-42c2-b4f3-63ecc3f7e318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:06.662807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42275acb-6d47-42c2-b4f3-63ecc3f7e318 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:06.663089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42275acb-6d47-42c2-b4f3-63ecc3f7e318 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:06.663362 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42275acb-6d47-42c2-b4f3-63ecc3f7e318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:06.665254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-42275acb-6d47-42c2-b4f3-63ecc3f7e318 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:05:06.676500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42275acb-6d47-42c2-b4f3-63ecc3f7e318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:06.678100 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42275acb-6d47-42c2-b4f3-63ecc3f7e318 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:06.687423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1291/2583] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:05:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:05:09.750796 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334485d9-040f-4e0e-8edd-1ef4537ef695 service swift] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:09.761181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334485d9-040f-4e0e-8edd-1ef4537ef695 service swift] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:09.765235 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e7005-772c-48f1-94f2-850425afb3fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:09.765926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e7005-772c-48f1-94f2-850425afb3fd service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:09.766197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e7005-772c-48f1-94f2-850425afb3fd service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:09.766458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e7005-772c-48f1-94f2-850425afb3fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:09.768740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-511e7005-772c-48f1-94f2-850425afb3fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:05:09.853494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e7005-772c-48f1-94f2-850425afb3fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:09.855151 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e7005-772c-48f1-94f2-850425afb3fd service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:09.866698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1293/2584] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:05:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:05:16.683311 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=6 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:05:19.861490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=30 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:05:46.860084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42275acb-6d47-42c2-b4f3-63ecc3f7e318 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:05:46.863756 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-42275acb-6d47-42c2-b4f3-63ecc3f7e318 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:05:46.881693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42275acb-6d47-42c2-b4f3-63ecc3f7e318 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:05:46.887973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa993d9e-3dd1-420b-875c-8e19a0fe61aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:05:46.888727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa993d9e-3dd1-420b-875c-8e19a0fe61aa service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:05:46.889060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa993d9e-3dd1-420b-875c-8e19a0fe61aa service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:05:46.889375 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa993d9e-3dd1-420b-875c-8e19a0fe61aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:05:46.891498 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-aa993d9e-3dd1-420b-875c-8e19a0fe61aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:05:46.932770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa993d9e-3dd1-420b-875c-8e19a0fe61aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:05:46.934494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa993d9e-3dd1-420b-875c-8e19a0fe61aa service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:05:46.957946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1292/2585] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:05:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:05:56.954587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=19 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:08.580847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e7005-772c-48f1-94f2-850425afb3fd service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:08.597350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e7005-772c-48f1-94f2-850425afb3fd service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:08.603987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded0f0d-7d5f-420e-876e-27296aeb6f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:08.604990 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded0f0d-7d5f-420e-876e-27296aeb6f63 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:08.605264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded0f0d-7d5f-420e-876e-27296aeb6f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:08.605598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded0f0d-7d5f-420e-876e-27296aeb6f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:08.607853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4ded0f0d-7d5f-420e-876e-27296aeb6f63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:08.696532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded0f0d-7d5f-420e-876e-27296aeb6f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:08.698240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded0f0d-7d5f-420e-876e-27296aeb6f63 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:08.710765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1294/2586] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:09.381996 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa993d9e-3dd1-420b-875c-8e19a0fe61aa service nova] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:09.396678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa993d9e-3dd1-420b-875c-8e19a0fe61aa service nova] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:09.402495 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5422fb9-e02d-4023-8f0d-62956a1d118d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:09.403434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5422fb9-e02d-4023-8f0d-62956a1d118d service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:09.403940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5422fb9-e02d-4023-8f0d-62956a1d118d service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:09.404315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5422fb9-e02d-4023-8f0d-62956a1d118d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:09.406576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d5422fb9-e02d-4023-8f0d-62956a1d118d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:09.421663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5422fb9-e02d-4023-8f0d-62956a1d118d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:09.423394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5422fb9-e02d-4023-8f0d-62956a1d118d service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:09.436139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1293/2587] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:13.053478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded0f0d-7d5f-420e-876e-27296aeb6f63 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:13.055142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-4ded0f0d-7d5f-420e-876e-27296aeb6f63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:13.161427 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5422fb9-e02d-4023-8f0d-62956a1d118d service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:13.163532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d5422fb9-e02d-4023-8f0d-62956a1d118d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:13.181482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded0f0d-7d5f-420e-876e-27296aeb6f63 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:13.185609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5422fb9-e02d-4023-8f0d-62956a1d118d service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:13.194917 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a39dea-7d8c-4bc2-879d-b0dc31a7b45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:13.196680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a39dea-7d8c-4bc2-879d-b0dc31a7b45c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:13.196903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5fb09-7036-4312-8c06-233c1abdd282 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:13.197139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a39dea-7d8c-4bc2-879d-b0dc31a7b45c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:13.197521 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a39dea-7d8c-4bc2-879d-b0dc31a7b45c admin admin] PATH_INFO: `/v3/users/1f026e791bf24a9fab1aaedb794e6229` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:13.197854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5fb09-7036-4312-8c06-233c1abdd282 service placement] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:13.198268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5fb09-7036-4312-8c06-233c1abdd282 service placement] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:13.198667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5fb09-7036-4312-8c06-233c1abdd282 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:13.199801 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a39dea-7d8c-4bc2-879d-b0dc31a7b45c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f026e791bf24a9fab1aaedb794e6229)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:13.201184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-dbd5fb09-7036-4312-8c06-233c1abdd282 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:13.217545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5fb09-7036-4312-8c06-233c1abdd282 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:13.219977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5fb09-7036-4312-8c06-233c1abdd282 service placement] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:13.220148 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a39dea-7d8c-4bc2-879d-b0dc31a7b45c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:13.252039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1295/2588] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:06:13.332844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b3a39dea-7d8c-4bc2-879d-b0dc31a7b45c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f026e791bf24a9fab1aaedb794e6229'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:13.343745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b3a39dea-7d8c-4bc2-879d-b0dc31a7b45c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f026e791bf24a9fab1aaedb794e6229'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:13.353211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b3a39dea-7d8c-4bc2-879d-b0dc31a7b45c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f026e791bf24a9fab1aaedb794e6229'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:13.357320 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b3a39dea-7d8c-4bc2-879d-b0dc31a7b45c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f026e791bf24a9fab1aaedb794e6229'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:13.376704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b3a39dea-7d8c-4bc2-879d-b0dc31a7b45c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f026e791bf24a9fab1aaedb794e6229'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:13.385746 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1294/2589] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:06:13 2022] DELETE /identity/v3/users/1f026e791bf24a9fab1aaedb794e6229 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:13.798857 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5fb09-7036-4312-8c06-233c1abdd282 service placement] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:13.826617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5fb09-7036-4312-8c06-233c1abdd282 service placement] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:13.965199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:13.967416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:13.968515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:13.969478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] PATH_INFO: `/v3/projects/c9432177a8ce4a4abec0ac6e372ce681` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:13.977525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9432177a8ce4a4abec0ac6e372ce681)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:13.991403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:14.062777 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:14.105369 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9432177a8ce4a4abec0ac6e372ce681', 'request_id': 'req-d8b96147-149d-43a4-9ef8-cc09360969c2'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:14.138471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9432177a8ce4a4abec0ac6e372ce681', 'request_id': 'req-d8b96147-149d-43a4-9ef8-cc09360969c2'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:14.154248 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9432177a8ce4a4abec0ac6e372ce681', 'request_id': 'req-d8b96147-149d-43a4-9ef8-cc09360969c2'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:14.184798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] The token cache is being invalidate because project c9432177a8ce4a4abec0ac6e372ce681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:06:14.185325 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:14.186632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-d8b96147-149d-43a4-9ef8-cc09360969c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:14.189630 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1296/2590] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:06:13 2022] DELETE /identity/v3/projects/c9432177a8ce4a4abec0ac6e372ce681 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:14.198740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f164b-ae53-4989-829b-4d3404f88da0 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:14.199554 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f164b-ae53-4989-829b-4d3404f88da0 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:14.199705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f164b-ae53-4989-829b-4d3404f88da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:14.246309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-315f164b-ae53-4989-829b-4d3404f88da0 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:14.247584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-315f164b-ae53-4989-829b-4d3404f88da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:14.345585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1295/2591] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:14.356730 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5ada6-70ff-4659-9044-ac363af643cd None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:14.357332 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5ada6-70ff-4659-9044-ac363af643cd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:14.358102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5ada6-70ff-4659-9044-ac363af643cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:14.383053 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-66f5ada6-70ff-4659-9044-ac363af643cd None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:14.384424 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-66f5ada6-70ff-4659-9044-ac363af643cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:14.403196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1297/2592] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:14.404643 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f164b-ae53-4989-829b-4d3404f88da0 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:14.419106 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f164b-ae53-4989-829b-4d3404f88da0 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:14.426896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831eb40-1447-4e3d-8a99-a8fd1e2e9e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:14.428105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831eb40-1447-4e3d-8a99-a8fd1e2e9e44 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:14.428450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831eb40-1447-4e3d-8a99-a8fd1e2e9e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:14.428711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831eb40-1447-4e3d-8a99-a8fd1e2e9e44 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:14.429502 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831eb40-1447-4e3d-8a99-a8fd1e2e9e44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:06:14.429816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831eb40-1447-4e3d-8a99-a8fd1e2e9e44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:14.431632 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831eb40-1447-4e3d-8a99-a8fd1e2e9e44 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:14.453881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1296/2593] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:06:14.455816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5ada6-70ff-4659-9044-ac363af643cd None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:14.466382 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5ada6-70ff-4659-9044-ac363af643cd None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:14.473682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d4975-71e0-46f2-9866-46d87c7a1d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:14.489059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d4975-71e0-46f2-9866-46d87c7a1d8a admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:14.489417 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d4975-71e0-46f2-9866-46d87c7a1d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:14.489654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d4975-71e0-46f2-9866-46d87c7a1d8a admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:14.490385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d4975-71e0-46f2-9866-46d87c7a1d8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:14.492479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d4975-71e0-46f2-9866-46d87c7a1d8a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:14.554605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1298/2594] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:06:14 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:14.560482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831eb40-1447-4e3d-8a99-a8fd1e2e9e44 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:14.575478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831eb40-1447-4e3d-8a99-a8fd1e2e9e44 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:14.611173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d4975-71e0-46f2-9866-46d87c7a1d8a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:14.613911 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d00d4975-71e0-46f2-9866-46d87c7a1d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:14.653176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:14.673774 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da3261-8edf-41b6-b125-9c3f866dbed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:14.675000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da3261-8edf-41b6-b125-9c3f866dbed6 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:14.675286 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da3261-8edf-41b6-b125-9c3f866dbed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:14.675569 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da3261-8edf-41b6-b125-9c3f866dbed6 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:14.676287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da3261-8edf-41b6-b125-9c3f866dbed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:14.678486 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da3261-8edf-41b6-b125-9c3f866dbed6 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:14.712436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1297/2595] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:06:14 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:14.717512 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da3261-8edf-41b6-b125-9c3f866dbed6 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:14.720585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d4975-71e0-46f2-9866-46d87c7a1d8a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:14.725671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bdf87-fa50-45eb-bd5a-ef7fb4f68514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:14.726647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bdf87-fa50-45eb-bd5a-ef7fb4f68514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:14.726647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bdf87-fa50-45eb-bd5a-ef7fb4f68514 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:14.727329 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bdf87-fa50-45eb-bd5a-ef7fb4f68514 admin admin] PATH_INFO: `/v3/users/d4c6ceff33ee449193fe6d4f7b2c01ec` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:14.728608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bdf87-fa50-45eb-bd5a-ef7fb4f68514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c6ceff33ee449193fe6d4f7b2c01ec)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:14.730746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bdf87-fa50-45eb-bd5a-ef7fb4f68514 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:14.739721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da3261-8edf-41b6-b125-9c3f866dbed6 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:14.748163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed9c05-e9e8-4ce1-858a-ee571d04ea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:14.750415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed9c05-e9e8-4ce1-858a-ee571d04ea21 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:14.751110 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed9c05-e9e8-4ce1-858a-ee571d04ea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:14.751381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed9c05-e9e8-4ce1-858a-ee571d04ea21 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:14.752900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed9c05-e9e8-4ce1-858a-ee571d04ea21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:14.755814 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed9c05-e9e8-4ce1-858a-ee571d04ea21 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:14.772301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1298/2596] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:14.777147 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed9c05-e9e8-4ce1-858a-ee571d04ea21 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:14.788652 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed9c05-e9e8-4ce1-858a-ee571d04ea21 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:14.795395 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65466524-b0de-4baa-a872-11cb860f5473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:14.796263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65466524-b0de-4baa-a872-11cb860f5473 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:14.796626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65466524-b0de-4baa-a872-11cb860f5473 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:14.797071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65466524-b0de-4baa-a872-11cb860f5473 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:14.798751 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65466524-b0de-4baa-a872-11cb860f5473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:14.801565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65466524-b0de-4baa-a872-11cb860f5473 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:14.803562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-620bdf87-fa50-45eb-bd5a-ef7fb4f68514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c6ceff33ee449193fe6d4f7b2c01ec'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:14.814231 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-620bdf87-fa50-45eb-bd5a-ef7fb4f68514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c6ceff33ee449193fe6d4f7b2c01ec'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:14.814967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-620bdf87-fa50-45eb-bd5a-ef7fb4f68514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c6ceff33ee449193fe6d4f7b2c01ec'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:14.820748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1299/2597] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:14.822002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65466524-b0de-4baa-a872-11cb860f5473 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:14.824008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-65466524-b0de-4baa-a872-11cb860f5473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:14.834052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-620bdf87-fa50-45eb-bd5a-ef7fb4f68514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c6ceff33ee449193fe6d4f7b2c01ec'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:14.841250 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:14.841779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65466524-b0de-4baa-a872-11cb860f5473 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:14.846135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-620bdf87-fa50-45eb-bd5a-ef7fb4f68514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c6ceff33ee449193fe6d4f7b2c01ec'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:14.848283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69c041-aaf8-4d74-857b-a45de87ac51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:14.849239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69c041-aaf8-4d74-857b-a45de87ac51a admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:14.853016 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69c041-aaf8-4d74-857b-a45de87ac51a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:14.853193 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69c041-aaf8-4d74-857b-a45de87ac51a admin admin] PATH_INFO: `/v3/projects/cb315d23f40a4d1bbe9828c1d3a4e5a7/users/187add5855314ff5b802b74297a6e7b0/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:14.864820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1299/2598] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:06:14 2022] DELETE /identity/v3/users/d4c6ceff33ee449193fe6d4f7b2c01ec => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:06:14.879195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69c041-aaf8-4d74-857b-a45de87ac51a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb315d23f40a4d1bbe9828c1d3a4e5a7, user_id=187add5855314ff5b802b74297a6e7b0, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:14.884644 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69c041-aaf8-4d74-857b-a45de87ac51a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:14.905180 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1300/2599] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:14 2022] PUT /identity/v3/projects/cb315d23f40a4d1bbe9828c1d3a4e5a7/users/187add5855314ff5b802b74297a6e7b0/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:14.912084 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3030ba-a0b3-4d28-90de-7a35b0e3df42 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:14.912418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3030ba-a0b3-4d28-90de-7a35b0e3df42 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:14.912850 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3030ba-a0b3-4d28-90de-7a35b0e3df42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:14.934819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-de3030ba-a0b3-4d28-90de-7a35b0e3df42 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:14.936071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-de3030ba-a0b3-4d28-90de-7a35b0e3df42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:15.027740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1300/2600] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:15.043465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69c041-aaf8-4d74-857b-a45de87ac51a admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:15.045672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3e69c041-aaf8-4d74-857b-a45de87ac51a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:15.106579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3030ba-a0b3-4d28-90de-7a35b0e3df42 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:15.108996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-de3030ba-a0b3-4d28-90de-7a35b0e3df42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:15.129609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69c041-aaf8-4d74-857b-a45de87ac51a admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:15.134127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3030ba-a0b3-4d28-90de-7a35b0e3df42 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:15.134728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf01815-0257-48cc-9e82-31e3ac11acf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:15.135516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf01815-0257-48cc-9e82-31e3ac11acf1 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:15.135882 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf01815-0257-48cc-9e82-31e3ac11acf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:15.136254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf01815-0257-48cc-9e82-31e3ac11acf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:15.145344 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:15.147611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:15.148093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:15.148596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] PATH_INFO: `/v3/projects/0c2f1232dd344ca7a48bf3fbc36dfb39` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:15.151292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c2f1232dd344ca7a48bf3fbc36dfb39)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:15.154100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:15.154229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf01815-0257-48cc-9e82-31e3ac11acf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:15.157591 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf01815-0257-48cc-9e82-31e3ac11acf1 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:15.177153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1301/2601] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:15.231025 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c2f1232dd344ca7a48bf3fbc36dfb39', 'request_id': 'req-b59bde98-b277-4fb1-8f08-531ef556e5e1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:15.241713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c2f1232dd344ca7a48bf3fbc36dfb39', 'request_id': 'req-b59bde98-b277-4fb1-8f08-531ef556e5e1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:15.252875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c2f1232dd344ca7a48bf3fbc36dfb39', 'request_id': 'req-b59bde98-b277-4fb1-8f08-531ef556e5e1'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:15.270338 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] The token cache is being invalidate because project 0c2f1232dd344ca7a48bf3fbc36dfb39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:06:15.270840 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:15.271804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-b59bde98-b277-4fb1-8f08-531ef556e5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:15.274807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1301/2602] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:06:15 2022] DELETE /identity/v3/projects/0c2f1232dd344ca7a48bf3fbc36dfb39 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:15.281468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07de71-31df-428d-8ccf-930814609d76 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:15.282382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07de71-31df-428d-8ccf-930814609d76 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:15.282549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07de71-31df-428d-8ccf-930814609d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:15.324278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-bb07de71-31df-428d-8ccf-930814609d76 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:15.326639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-bb07de71-31df-428d-8ccf-930814609d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:15.442690 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1302/2603] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:15.451985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce77aa-3b01-4627-99e6-01d48581fcef None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:15.452319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce77aa-3b01-4627-99e6-01d48581fcef None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:15.452619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce77aa-3b01-4627-99e6-01d48581fcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:15.475808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-b2ce77aa-3b01-4627-99e6-01d48581fcef None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:15.477546 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b2ce77aa-3b01-4627-99e6-01d48581fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:15.491160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1302/2604] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:06:15.493231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07de71-31df-428d-8ccf-930814609d76 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:15.511811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07de71-31df-428d-8ccf-930814609d76 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:15.521539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d758bb-edce-4138-aab6-6740ca00db9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:15.522700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d758bb-edce-4138-aab6-6740ca00db9d admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:15.523239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d758bb-edce-4138-aab6-6740ca00db9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:15.523784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d758bb-edce-4138-aab6-6740ca00db9d admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:15.525528 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d758bb-edce-4138-aab6-6740ca00db9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:06:15.526777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d758bb-edce-4138-aab6-6740ca00db9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:15.529854 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d758bb-edce-4138-aab6-6740ca00db9d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:15.556193 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1303/2605] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:06:15.563031 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce77aa-3b01-4627-99e6-01d48581fcef None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:15.584971 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce77aa-3b01-4627-99e6-01d48581fcef None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:15.591415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269e7a3-cbe7-4885-a324-cd6e9db7c636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:15.592459 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e7a3-cbe7-4885-a324-cd6e9db7c636 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:15.592926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e7a3-cbe7-4885-a324-cd6e9db7c636 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:15.593419 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269e7a3-cbe7-4885-a324-cd6e9db7c636 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:15.594707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269e7a3-cbe7-4885-a324-cd6e9db7c636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:15.598984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269e7a3-cbe7-4885-a324-cd6e9db7c636 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:15.654245 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1303/2606] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:06:15 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:06:15.658353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d758bb-edce-4138-aab6-6740ca00db9d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:15.678469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d758bb-edce-4138-aab6-6740ca00db9d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:15.709720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:15.778562 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a6aeb-d954-45c9-a7ba-11c0a81b55d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:15.780212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a6aeb-d954-45c9-a7ba-11c0a81b55d3 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:15.780759 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a6aeb-d954-45c9-a7ba-11c0a81b55d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:15.781265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a6aeb-d954-45c9-a7ba-11c0a81b55d3 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:15.782561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a6aeb-d954-45c9-a7ba-11c0a81b55d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:15.787036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a6aeb-d954-45c9-a7ba-11c0a81b55d3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:15.821037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1304/2607] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:06:15 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:15.826627 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269e7a3-cbe7-4885-a324-cd6e9db7c636 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:15.841277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269e7a3-cbe7-4885-a324-cd6e9db7c636 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:15.850173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f0372-fa0c-45f2-bbb2-74fdc95a4400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:15.851405 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f0372-fa0c-45f2-bbb2-74fdc95a4400 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:15.851820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f0372-fa0c-45f2-bbb2-74fdc95a4400 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:15.852255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f0372-fa0c-45f2-bbb2-74fdc95a4400 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:15.853383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f0372-fa0c-45f2-bbb2-74fdc95a4400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:15.855388 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f0372-fa0c-45f2-bbb2-74fdc95a4400 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:15.869385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1304/2608] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:15.874051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a6aeb-d954-45c9-a7ba-11c0a81b55d3 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:15.889777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a6aeb-d954-45c9-a7ba-11c0a81b55d3 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:15.895600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9794e6-e5ca-4a09-8ea5-a2ca26263398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:15.896490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9794e6-e5ca-4a09-8ea5-a2ca26263398 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:15.896865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9794e6-e5ca-4a09-8ea5-a2ca26263398 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:15.897235 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9794e6-e5ca-4a09-8ea5-a2ca26263398 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:15.898841 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9794e6-e5ca-4a09-8ea5-a2ca26263398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:15.900633 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9794e6-e5ca-4a09-8ea5-a2ca26263398 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:15.916395 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1305/2609] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:06:15.918271 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f0372-fa0c-45f2-bbb2-74fdc95a4400 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:15.929384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f0372-fa0c-45f2-bbb2-74fdc95a4400 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:15.934701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c88dc8-cf2b-4ce3-abcf-7e48e2d655d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:15.935630 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88dc8-cf2b-4ce3-abcf-7e48e2d655d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:15.935919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88dc8-cf2b-4ce3-abcf-7e48e2d655d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:15.936198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88dc8-cf2b-4ce3-abcf-7e48e2d655d6 admin admin] PATH_INFO: `/v3/projects/67f9686b49d54daab5ebeba4b25409a5/users/d6638deff67c4a009b5afb82e2b0633e/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:15.963134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c88dc8-cf2b-4ce3-abcf-7e48e2d655d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67f9686b49d54daab5ebeba4b25409a5, user_id=d6638deff67c4a009b5afb82e2b0633e, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:15.966035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c88dc8-cf2b-4ce3-abcf-7e48e2d655d6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:15.986140 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1305/2610] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:15 2022] PUT /identity/v3/projects/67f9686b49d54daab5ebeba4b25409a5/users/d6638deff67c4a009b5afb82e2b0633e/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:15.998907 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24b8bf-9138-4aa5-8ad1-f3a3836fc13d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:16.000685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24b8bf-9138-4aa5-8ad1-f3a3836fc13d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:16.001184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24b8bf-9138-4aa5-8ad1-f3a3836fc13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:16.068863 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-7b24b8bf-9138-4aa5-8ad1-f3a3836fc13d None None] MFA Rules not processed for user `d6638deff67c4a009b5afb82e2b0633e`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:16.071388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7b24b8bf-9138-4aa5-8ad1-f3a3836fc13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:16.223647 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1306/2611] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:16.230680 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c88dc8-cf2b-4ce3-abcf-7e48e2d655d6 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:16.258889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:16.332518 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c88dc8-cf2b-4ce3-abcf-7e48e2d655d6 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:16.337830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049860d-8bd9-4fd3-b1c6-2285e862cbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:16.338629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049860d-8bd9-4fd3-b1c6-2285e862cbb7 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:16.338987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049860d-8bd9-4fd3-b1c6-2285e862cbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:16.339390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049860d-8bd9-4fd3-b1c6-2285e862cbb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:16.340183 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049860d-8bd9-4fd3-b1c6-2285e862cbb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:16.342410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049860d-8bd9-4fd3-b1c6-2285e862cbb7 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:16.387498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1306/2612] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:06:16 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:16.389953 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24b8bf-9138-4aa5-8ad1-f3a3836fc13d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:16.415729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24b8bf-9138-4aa5-8ad1-f3a3836fc13d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:16.529724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c201e3-a79a-4317-b31a-0a234c7d545f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:16.530541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c201e3-a79a-4317-b31a-0a234c7d545f admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:16.530895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c201e3-a79a-4317-b31a-0a234c7d545f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:16.531261 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c201e3-a79a-4317-b31a-0a234c7d545f admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:16.532029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c201e3-a79a-4317-b31a-0a234c7d545f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:16.534646 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c201e3-a79a-4317-b31a-0a234c7d545f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:16.561672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1307/2613] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:06:16 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:16.567607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049860d-8bd9-4fd3-b1c6-2285e862cbb7 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:16.586242 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049860d-8bd9-4fd3-b1c6-2285e862cbb7 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:16.594476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc338b-475e-44f2-acaa-8aee8266c11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:16.596541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc338b-475e-44f2-acaa-8aee8266c11e admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:16.596541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc338b-475e-44f2-acaa-8aee8266c11e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:16.597118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc338b-475e-44f2-acaa-8aee8266c11e admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:16.598952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc338b-475e-44f2-acaa-8aee8266c11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:16.601884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc338b-475e-44f2-acaa-8aee8266c11e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:16.620109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1307/2614] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:16.623851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c201e3-a79a-4317-b31a-0a234c7d545f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:16.645877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c201e3-a79a-4317-b31a-0a234c7d545f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:16.655431 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aeeaf3-071b-46ee-88e9-9557b9b959c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:16.657235 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeeaf3-071b-46ee-88e9-9557b9b959c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:16.657595 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeeaf3-071b-46ee-88e9-9557b9b959c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:16.658099 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeeaf3-071b-46ee-88e9-9557b9b959c4 admin admin] PATH_INFO: `/v3/projects/052b5027ae044baba55363ac2759423e/users/3c98dd833f604d098e4a7955b99c1b37/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:16.702674 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aeeaf3-071b-46ee-88e9-9557b9b959c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=052b5027ae044baba55363ac2759423e, user_id=3c98dd833f604d098e4a7955b99c1b37, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:16.708251 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aeeaf3-071b-46ee-88e9-9557b9b959c4 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:16.711744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:16.736266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1308/2615] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:16 2022] PUT /identity/v3/projects/052b5027ae044baba55363ac2759423e/users/3c98dd833f604d098e4a7955b99c1b37/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:16.741193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc338b-475e-44f2-acaa-8aee8266c11e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:16.761766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc338b-475e-44f2-acaa-8aee8266c11e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:16.876267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1a2b9-6270-4406-b811-7b588af7130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:16.876825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1a2b9-6270-4406-b811-7b588af7130c admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:16.877384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1a2b9-6270-4406-b811-7b588af7130c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:16.877847 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1a2b9-6270-4406-b811-7b588af7130c admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:16.879381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1a2b9-6270-4406-b811-7b588af7130c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:16.881646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1a2b9-6270-4406-b811-7b588af7130c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:16.905095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1308/2616] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:16.907249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aeeaf3-071b-46ee-88e9-9557b9b959c4 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:16.919873 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aeeaf3-071b-46ee-88e9-9557b9b959c4 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:16.926537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c2177-1ea0-4fda-bc56-a8a9f765bf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:16.927961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c2177-1ea0-4fda-bc56-a8a9f765bf1a admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:16.927961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c2177-1ea0-4fda-bc56-a8a9f765bf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:16.928178 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c2177-1ea0-4fda-bc56-a8a9f765bf1a admin admin] PATH_INFO: `/v3/projects/052b5027ae044baba55363ac2759423e/users/3c98dd833f604d098e4a7955b99c1b37/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:16.930239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c2177-1ea0-4fda-bc56-a8a9f765bf1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=052b5027ae044baba55363ac2759423e, user_id=3c98dd833f604d098e4a7955b99c1b37, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:16.933003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c2177-1ea0-4fda-bc56-a8a9f765bf1a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:16.947623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1309/2617] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:16 2022] PUT /identity/v3/projects/052b5027ae044baba55363ac2759423e/users/3c98dd833f604d098e4a7955b99c1b37/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:16.958853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f3903-37ae-4a7e-9c7b-375f5eb808ba None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:16.959244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f3903-37ae-4a7e-9c7b-375f5eb808ba None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:16.959441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f3903-37ae-4a7e-9c7b-375f5eb808ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:16.996580 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ee1f3903-37ae-4a7e-9c7b-375f5eb808ba None None] MFA Rules not processed for user `3c98dd833f604d098e4a7955b99c1b37`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:16.997306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ee1f3903-37ae-4a7e-9c7b-375f5eb808ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:17.103337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1309/2618] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:06:17.117427 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c2177-1ea0-4fda-bc56-a8a9f765bf1a admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:17.121109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-109c2177-1ea0-4fda-bc56-a8a9f765bf1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:17.252880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c2177-1ea0-4fda-bc56-a8a9f765bf1a admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:17.264289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6197c87-0200-44cb-adf6-40e44cddc65b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:17.265577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6197c87-0200-44cb-adf6-40e44cddc65b service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:17.266118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6197c87-0200-44cb-adf6-40e44cddc65b service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:17.266780 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6197c87-0200-44cb-adf6-40e44cddc65b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:17.289090 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6197c87-0200-44cb-adf6-40e44cddc65b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:17.293416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6197c87-0200-44cb-adf6-40e44cddc65b service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:17.328376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1310/2619] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:17.472808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f3903-37ae-4a7e-9c7b-375f5eb808ba None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:17.485506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f3903-37ae-4a7e-9c7b-375f5eb808ba None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:17.489618 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f79739-90a1-4b45-864c-b1514a7935a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:17.490296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f79739-90a1-4b45-864c-b1514a7935a0 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:17.490587 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f79739-90a1-4b45-864c-b1514a7935a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:17.490872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f79739-90a1-4b45-864c-b1514a7935a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:17.501650 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:17.578591 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f79739-90a1-4b45-864c-b1514a7935a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:17.581226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f79739-90a1-4b45-864c-b1514a7935a0 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:17.601387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1310/2620] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:21.735088 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430297a-fc61-4a43-8323-6a3e421a5667 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:21.735685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430297a-fc61-4a43-8323-6a3e421a5667 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:21.735685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430297a-fc61-4a43-8323-6a3e421a5667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:21.778662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-1430297a-fc61-4a43-8323-6a3e421a5667 None None] MFA Rules not processed for user `187add5855314ff5b802b74297a6e7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:21.780051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1430297a-fc61-4a43-8323-6a3e421a5667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:21.865095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1311/2621] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:06:21.886944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f79739-90a1-4b45-864c-b1514a7935a0 service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:21.889810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e0f79739-90a1-4b45-864c-b1514a7935a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:22.048163 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f79739-90a1-4b45-864c-b1514a7935a0 service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:22.057134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e44e23-8d62-44a1-b33b-d608afaafc63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:22.058141 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e44e23-8d62-44a1-b33b-d608afaafc63 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:22.058779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e44e23-8d62-44a1-b33b-d608afaafc63 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:22.059122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e44e23-8d62-44a1-b33b-d608afaafc63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:22.074032 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e44e23-8d62-44a1-b33b-d608afaafc63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:22.081725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e44e23-8d62-44a1-b33b-d608afaafc63 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:22.100598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1311/2622] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:23.901865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac61b3-f844-4d23-9024-10a734b12039 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:23.902556 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac61b3-f844-4d23-9024-10a734b12039 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:23.902868 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac61b3-f844-4d23-9024-10a734b12039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:23.928354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-eeac61b3-f844-4d23-9024-10a734b12039 None None] MFA Rules not processed for user `187add5855314ff5b802b74297a6e7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:23.929773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-eeac61b3-f844-4d23-9024-10a734b12039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:23.937973 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1312/2623] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:25.222748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e44e23-8d62-44a1-b33b-d608afaafc63 service nova] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:25.224854 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-25e44e23-8d62-44a1-b33b-d608afaafc63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:25.353407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e44e23-8d62-44a1-b33b-d608afaafc63 service nova] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:25.367347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334ab3f-23e5-419b-b239-3e49efe2b574 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:25.368685 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334ab3f-23e5-419b-b239-3e49efe2b574 service placement] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:25.369184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334ab3f-23e5-419b-b239-3e49efe2b574 service placement] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:25.369661 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334ab3f-23e5-419b-b239-3e49efe2b574 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:25.373298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-5334ab3f-23e5-419b-b239-3e49efe2b574 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:25.401030 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334ab3f-23e5-419b-b239-3e49efe2b574 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:25.404160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334ab3f-23e5-419b-b239-3e49efe2b574 service placement] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:25.427808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1312/2624] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:29.958294 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac61b3-f844-4d23-9024-10a734b12039 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:30.102365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac61b3-f844-4d23-9024-10a734b12039 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:30.112518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:30.117763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:30.117976 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:30.118417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] PATH_INFO: `/v3/users/d6638deff67c4a009b5afb82e2b0633e` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:30.120065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6638deff67c4a009b5afb82e2b0633e)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:30.122533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:30.140160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:30.280607 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6638deff67c4a009b5afb82e2b0633e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:30.304817 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6638deff67c4a009b5afb82e2b0633e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:30.336159 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6638deff67c4a009b5afb82e2b0633e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:30.339465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6638deff67c4a009b5afb82e2b0633e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:30.376708 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6638deff67c4a009b5afb82e2b0633e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:30.391994 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1313/2625] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:06:29 2022] DELETE /identity/v3/users/d6638deff67c4a009b5afb82e2b0633e => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:30.400394 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f231f-de9e-4e1a-8f44-c2cd2b907d51 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:30.401998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f231f-de9e-4e1a-8f44-c2cd2b907d51 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:30.402593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f231f-de9e-4e1a-8f44-c2cd2b907d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:30.441640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-566f231f-de9e-4e1a-8f44-c2cd2b907d51 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:30.445631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-566f231f-de9e-4e1a-8f44-c2cd2b907d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:30.486907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:30.563876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1313/2626] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:30.581041 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:30.583784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:30.695960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc444d-925a-457a-81c9-a7f154808140 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:30.708582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd8363-ddab-4dde-a968-3a86b7eb4188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:30.709626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd8363-ddab-4dde-a968-3a86b7eb4188 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:30.709956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd8363-ddab-4dde-a968-3a86b7eb4188 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:30.710398 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd8363-ddab-4dde-a968-3a86b7eb4188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:30.728164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd8363-ddab-4dde-a968-3a86b7eb4188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:30.731304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd8363-ddab-4dde-a968-3a86b7eb4188 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:30.754388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1314/2627] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:31.223864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f231f-de9e-4e1a-8f44-c2cd2b907d51 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:31.226300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-566f231f-de9e-4e1a-8f44-c2cd2b907d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:31.246211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f231f-de9e-4e1a-8f44-c2cd2b907d51 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:31.256159 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:31.257243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:31.257896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:31.258413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] PATH_INFO: `/v3/projects/67f9686b49d54daab5ebeba4b25409a5` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:31.261421 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=67f9686b49d54daab5ebeba4b25409a5)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:31.266019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:31.352102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67f9686b49d54daab5ebeba4b25409a5', 'request_id': 'req-01fa0869-5828-4202-8e4e-9d007ba1c0cf'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:31.367028 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67f9686b49d54daab5ebeba4b25409a5', 'request_id': 'req-01fa0869-5828-4202-8e4e-9d007ba1c0cf'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:31.380356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67f9686b49d54daab5ebeba4b25409a5', 'request_id': 'req-01fa0869-5828-4202-8e4e-9d007ba1c0cf'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:31.393720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] The token cache is being invalidate because project 67f9686b49d54daab5ebeba4b25409a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:06:31.394059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:31.394742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:31.396428 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1314/2628] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:06:31 2022] DELETE /identity/v3/projects/67f9686b49d54daab5ebeba4b25409a5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:31.401389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd8363-ddab-4dde-a968-3a86b7eb4188 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:31.419710 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd8363-ddab-4dde-a968-3a86b7eb4188 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:31.525705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f87ce-621d-4e6e-960c-45c8abc46cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:31.527276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f87ce-621d-4e6e-960c-45c8abc46cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:31.527776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f87ce-621d-4e6e-960c-45c8abc46cec admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:31.528267 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f87ce-621d-4e6e-960c-45c8abc46cec admin admin] PATH_INFO: `/v3/users/3c98dd833f604d098e4a7955b99c1b37` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:31.530760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f87ce-621d-4e6e-960c-45c8abc46cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c98dd833f604d098e4a7955b99c1b37)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:31.535164 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f87ce-621d-4e6e-960c-45c8abc46cec admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:31.600891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:31.629825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b27f87ce-621d-4e6e-960c-45c8abc46cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c98dd833f604d098e4a7955b99c1b37'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:31.641483 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b27f87ce-621d-4e6e-960c-45c8abc46cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c98dd833f604d098e4a7955b99c1b37'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:31.656128 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b27f87ce-621d-4e6e-960c-45c8abc46cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c98dd833f604d098e4a7955b99c1b37'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:31.657399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b27f87ce-621d-4e6e-960c-45c8abc46cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c98dd833f604d098e4a7955b99c1b37'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:31.686643 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b27f87ce-621d-4e6e-960c-45c8abc46cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c98dd833f604d098e4a7955b99c1b37'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:31.699661 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1315/2629] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:06:31 2022] DELETE /identity/v3/users/3c98dd833f604d098e4a7955b99c1b37 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:32.225110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:32.227671 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:32.246150 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa0869-5828-4202-8e4e-9d007ba1c0cf admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:32.363807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:32.365395 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:32.365896 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:32.366390 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] PATH_INFO: `/v3/projects/052b5027ae044baba55363ac2759423e` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:32.368878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] RBAC: Authorizing `identity:delete_project(project_id=052b5027ae044baba55363ac2759423e)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:32.372925 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:32.460357 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '052b5027ae044baba55363ac2759423e', 'request_id': 'req-627a76d1-8251-44c2-ad7c-1cc9355faacb'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:32.462979 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:32.473323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '052b5027ae044baba55363ac2759423e', 'request_id': 'req-627a76d1-8251-44c2-ad7c-1cc9355faacb'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:32.485397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '052b5027ae044baba55363ac2759423e', 'request_id': 'req-627a76d1-8251-44c2-ad7c-1cc9355faacb'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:32.499984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] The token cache is being invalidate because project 052b5027ae044baba55363ac2759423e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:06:32.500725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:32.501528 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-627a76d1-8251-44c2-ad7c-1cc9355faacb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:32.505110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1315/2630] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:06:32 2022] DELETE /identity/v3/projects/052b5027ae044baba55363ac2759423e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:32.513834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110235e-4cd4-417a-9130-12e79fe05712 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:32.514662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110235e-4cd4-417a-9130-12e79fe05712 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:32.514662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110235e-4cd4-417a-9130-12e79fe05712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:32.539722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-6110235e-4cd4-417a-9130-12e79fe05712 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:32.541237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6110235e-4cd4-417a-9130-12e79fe05712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:32.615712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1316/2631] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:32.626011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f17f-d282-486b-8940-f3b710d96204 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:32.626699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f17f-d282-486b-8940-f3b710d96204 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:32.627099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697f17f-d282-486b-8940-f3b710d96204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:32.648507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-a697f17f-d282-486b-8940-f3b710d96204 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:32.649874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a697f17f-d282-486b-8940-f3b710d96204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:32.659599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1316/2632] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:32.666661 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110235e-4cd4-417a-9130-12e79fe05712 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:32.685532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110235e-4cd4-417a-9130-12e79fe05712 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:32.693070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc2774-af62-4bc9-96d4-a14139507b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:32.693956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc2774-af62-4bc9-96d4-a14139507b35 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:32.694584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc2774-af62-4bc9-96d4-a14139507b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:32.694912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc2774-af62-4bc9-96d4-a14139507b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:32.695752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc2774-af62-4bc9-96d4-a14139507b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:06:32.696112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc2774-af62-4bc9-96d4-a14139507b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:32.698118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc2774-af62-4bc9-96d4-a14139507b35 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:32.712698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1317/2633] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:06:32.717522 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f17f-d282-486b-8940-f3b710d96204 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:32.729327 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697f17f-d282-486b-8940-f3b710d96204 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:32.734579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc6187-180f-4615-aa35-ddcf343d2812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:32.735373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc6187-180f-4615-aa35-ddcf343d2812 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:32.735713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc6187-180f-4615-aa35-ddcf343d2812 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:32.736036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc6187-180f-4615-aa35-ddcf343d2812 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:32.737402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc6187-180f-4615-aa35-ddcf343d2812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:32.740203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc6187-180f-4615-aa35-ddcf343d2812 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:32.793521 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1317/2634] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:06:32 2022] POST /identity/v3/projects => generated 387 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:06:32.799260 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc2774-af62-4bc9-96d4-a14139507b35 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:32.816185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc2774-af62-4bc9-96d4-a14139507b35 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:32.899653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf46ba3-dc16-4694-800a-85dcba072e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:32.900443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46ba3-dc16-4694-800a-85dcba072e92 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:32.900730 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46ba3-dc16-4694-800a-85dcba072e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:32.901008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46ba3-dc16-4694-800a-85dcba072e92 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:32.901786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf46ba3-dc16-4694-800a-85dcba072e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:32.904522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf46ba3-dc16-4694-800a-85dcba072e92 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:32.932732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1318/2635] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:06:32 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:32.937694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc6187-180f-4615-aa35-ddcf343d2812 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:32.950314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc6187-180f-4615-aa35-ddcf343d2812 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:32.956835 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd31b1-4fcb-4f70-94f9-c90f188fcefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:32.957184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd31b1-4fcb-4f70-94f9-c90f188fcefd admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:32.957334 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd31b1-4fcb-4f70-94f9-c90f188fcefd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:32.957944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd31b1-4fcb-4f70-94f9-c90f188fcefd admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:32.958663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd31b1-4fcb-4f70-94f9-c90f188fcefd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:32.959960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddd31b1-4fcb-4f70-94f9-c90f188fcefd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:32.973709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1318/2636] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:32.980604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf46ba3-dc16-4694-800a-85dcba072e92 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:33.000056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf46ba3-dc16-4694-800a-85dcba072e92 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:33.007321 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52e112-6459-4fc8-8476-9b79c212d23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:33.008399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52e112-6459-4fc8-8476-9b79c212d23a admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:33.008904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52e112-6459-4fc8-8476-9b79c212d23a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:33.009210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52e112-6459-4fc8-8476-9b79c212d23a admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:33.010012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52e112-6459-4fc8-8476-9b79c212d23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:33.011761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52e112-6459-4fc8-8476-9b79c212d23a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:33.024597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1319/2637] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:33.029348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd31b1-4fcb-4f70-94f9-c90f188fcefd admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:33.043678 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd31b1-4fcb-4f70-94f9-c90f188fcefd admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:33.050584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12ee9c-a6ba-468e-8f73-a56f7b03f8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:33.051882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12ee9c-a6ba-468e-8f73-a56f7b03f8d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:33.052354 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12ee9c-a6ba-468e-8f73-a56f7b03f8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:33.052957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12ee9c-a6ba-468e-8f73-a56f7b03f8d8 admin admin] PATH_INFO: `/v3/projects/bdef8f59891044dfb7baa8647b764efd/users/2921c88157614e52bd26d955b5469fc6/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:33.075428 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12ee9c-a6ba-468e-8f73-a56f7b03f8d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdef8f59891044dfb7baa8647b764efd, user_id=2921c88157614e52bd26d955b5469fc6, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:33.078598 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12ee9c-a6ba-468e-8f73-a56f7b03f8d8 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:33.102581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1319/2638] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:33 2022] PUT /identity/v3/projects/bdef8f59891044dfb7baa8647b764efd/users/2921c88157614e52bd26d955b5469fc6/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:33.116928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851ed02-fe66-45d9-803a-ae2b903074ae None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:33.117468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851ed02-fe66-45d9-803a-ae2b903074ae None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:33.117999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851ed02-fe66-45d9-803a-ae2b903074ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:33.167721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:33.175805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d851ed02-fe66-45d9-803a-ae2b903074ae None None] MFA Rules not processed for user `2921c88157614e52bd26d955b5469fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:33.179315 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d851ed02-fe66-45d9-803a-ae2b903074ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:33.289791 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1320/2639] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:33.305663 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12ee9c-a6ba-468e-8f73-a56f7b03f8d8 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:33.429494 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12ee9c-a6ba-468e-8f73-a56f7b03f8d8 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:33.445417 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef816a-1cf8-4a5e-9254-788284f5fd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:33.446600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef816a-1cf8-4a5e-9254-788284f5fd7c admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:33.447075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef816a-1cf8-4a5e-9254-788284f5fd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:33.447513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef816a-1cf8-4a5e-9254-788284f5fd7c admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:33.448746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef816a-1cf8-4a5e-9254-788284f5fd7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:33.452026 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef816a-1cf8-4a5e-9254-788284f5fd7c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:33.505871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1320/2640] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:06:33 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:33.512860 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851ed02-fe66-45d9-803a-ae2b903074ae None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:33.534449 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851ed02-fe66-45d9-803a-ae2b903074ae None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:33.651819 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb32dc-0507-4e0a-ba9d-003fc0d4651c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:33.652987 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb32dc-0507-4e0a-ba9d-003fc0d4651c admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:33.653457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb32dc-0507-4e0a-ba9d-003fc0d4651c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:33.653923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb32dc-0507-4e0a-ba9d-003fc0d4651c admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:33.655124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb32dc-0507-4e0a-ba9d-003fc0d4651c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:33.659347 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb32dc-0507-4e0a-ba9d-003fc0d4651c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:33.694552 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1321/2641] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:06:33 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:33.699525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef816a-1cf8-4a5e-9254-788284f5fd7c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:33.712764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef816a-1cf8-4a5e-9254-788284f5fd7c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:33.718977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8af70d-02bb-47ce-a5c5-13e94e56fba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:33.719899 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8af70d-02bb-47ce-a5c5-13e94e56fba8 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:33.720375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8af70d-02bb-47ce-a5c5-13e94e56fba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:33.721336 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8af70d-02bb-47ce-a5c5-13e94e56fba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:33.722237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8af70d-02bb-47ce-a5c5-13e94e56fba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:33.724168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8af70d-02bb-47ce-a5c5-13e94e56fba8 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:33.744482 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1321/2642] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:33.746655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb32dc-0507-4e0a-ba9d-003fc0d4651c admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:33.761823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb32dc-0507-4e0a-ba9d-003fc0d4651c admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:33.770020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d3b18-936c-4e8c-8441-3e69f891de19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:33.770866 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d3b18-936c-4e8c-8441-3e69f891de19 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:33.771256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d3b18-936c-4e8c-8441-3e69f891de19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:33.771487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d3b18-936c-4e8c-8441-3e69f891de19 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:33.772459 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d3b18-936c-4e8c-8441-3e69f891de19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:33.774196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d3b18-936c-4e8c-8441-3e69f891de19 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:33.788131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1322/2643] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:33.795394 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8af70d-02bb-47ce-a5c5-13e94e56fba8 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:33.810039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8af70d-02bb-47ce-a5c5-13e94e56fba8 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:33.816047 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762a7a4-84ac-402a-9e8c-157a16a4d9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:33.817078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762a7a4-84ac-402a-9e8c-157a16a4d9af admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:33.817364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762a7a4-84ac-402a-9e8c-157a16a4d9af admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:33.817646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762a7a4-84ac-402a-9e8c-157a16a4d9af admin admin] PATH_INFO: `/v3/projects/a78f3ba51c2b42c187eb1c7174ec3f13/users/0376bdc93fff468ea7a3b5fc4087cce4/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:33.836408 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762a7a4-84ac-402a-9e8c-157a16a4d9af admin admin] RBAC: Authorizing `identity:create_grant(project_id=a78f3ba51c2b42c187eb1c7174ec3f13, user_id=0376bdc93fff468ea7a3b5fc4087cce4, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:33.840082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762a7a4-84ac-402a-9e8c-157a16a4d9af admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:33.863435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1322/2644] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:33 2022] PUT /identity/v3/projects/a78f3ba51c2b42c187eb1c7174ec3f13/users/0376bdc93fff468ea7a3b5fc4087cce4/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:33.880897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ed3f7-99ef-49af-96c1-17b1e591ae45 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:33.881360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ed3f7-99ef-49af-96c1-17b1e591ae45 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:33.881805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ed3f7-99ef-49af-96c1-17b1e591ae45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:33.918979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-f07ed3f7-99ef-49af-96c1-17b1e591ae45 None None] MFA Rules not processed for user `0376bdc93fff468ea7a3b5fc4087cce4`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:33.920538 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f07ed3f7-99ef-49af-96c1-17b1e591ae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:34.005529 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1323/2645] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:34.011039 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762a7a4-84ac-402a-9e8c-157a16a4d9af admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:34.058644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:34.119148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762a7a4-84ac-402a-9e8c-157a16a4d9af admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:34.125722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ad24d-cbc7-4756-89e5-1ac2687d8e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:34.127019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ad24d-cbc7-4756-89e5-1ac2687d8e63 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:34.127430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ad24d-cbc7-4756-89e5-1ac2687d8e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:34.127893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ad24d-cbc7-4756-89e5-1ac2687d8e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:34.129169 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ad24d-cbc7-4756-89e5-1ac2687d8e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:34.132647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ad24d-cbc7-4756-89e5-1ac2687d8e63 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:34.177208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1323/2646] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:06:34 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:34.182283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ed3f7-99ef-49af-96c1-17b1e591ae45 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:34.197033 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ed3f7-99ef-49af-96c1-17b1e591ae45 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:34.218452 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:34.289215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98168d48-55c9-4739-a0fb-31bc979495da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:34.290407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98168d48-55c9-4739-a0fb-31bc979495da admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:34.290773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98168d48-55c9-4739-a0fb-31bc979495da admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:34.291213 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98168d48-55c9-4739-a0fb-31bc979495da admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:34.292587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98168d48-55c9-4739-a0fb-31bc979495da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:34.296503 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98168d48-55c9-4739-a0fb-31bc979495da admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:34.351029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1324/2647] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:06:34 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:34.353208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ad24d-cbc7-4756-89e5-1ac2687d8e63 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:34.373353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ad24d-cbc7-4756-89e5-1ac2687d8e63 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:34.382110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7b339-7d18-4b00-b281-3115c1537b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:34.383268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7b339-7d18-4b00-b281-3115c1537b79 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:34.383771 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7b339-7d18-4b00-b281-3115c1537b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:34.384275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7b339-7d18-4b00-b281-3115c1537b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:34.385748 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7b339-7d18-4b00-b281-3115c1537b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:34.389037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7b339-7d18-4b00-b281-3115c1537b79 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:34.411058 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1324/2648] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:34.416153 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98168d48-55c9-4739-a0fb-31bc979495da admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:34.429021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98168d48-55c9-4739-a0fb-31bc979495da admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:34.434009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce06587-eef0-448f-ba5b-2219b67dfab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:34.434926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce06587-eef0-448f-ba5b-2219b67dfab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:34.435255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce06587-eef0-448f-ba5b-2219b67dfab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:34.435584 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce06587-eef0-448f-ba5b-2219b67dfab6 admin admin] PATH_INFO: `/v3/projects/8a61f14f37ec4a9eb97a1d069eb5ddcf/users/ee06ea0086014295849d3df1528dc2cd/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:34.456530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce06587-eef0-448f-ba5b-2219b67dfab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a61f14f37ec4a9eb97a1d069eb5ddcf, user_id=ee06ea0086014295849d3df1528dc2cd, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:34.459592 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce06587-eef0-448f-ba5b-2219b67dfab6 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:34.476114 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1325/2649] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:34 2022] PUT /identity/v3/projects/8a61f14f37ec4a9eb97a1d069eb5ddcf/users/ee06ea0086014295849d3df1528dc2cd/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:06:34.480782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7b339-7d18-4b00-b281-3115c1537b79 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:34.492961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7b339-7d18-4b00-b281-3115c1537b79 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:34.577628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b963f-928f-4996-b0ba-9192aab1b63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:34.578328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b963f-928f-4996-b0ba-9192aab1b63a admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:34.578609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b963f-928f-4996-b0ba-9192aab1b63a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:34.578894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b963f-928f-4996-b0ba-9192aab1b63a admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:34.579650 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b963f-928f-4996-b0ba-9192aab1b63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:34.581421 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b963f-928f-4996-b0ba-9192aab1b63a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:34.592945 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1325/2650] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:34.598450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce06587-eef0-448f-ba5b-2219b67dfab6 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:34.609636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce06587-eef0-448f-ba5b-2219b67dfab6 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:34.615141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ac654-481f-4e8c-8420-99529786c313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:34.616285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ac654-481f-4e8c-8420-99529786c313 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:34.616607 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ac654-481f-4e8c-8420-99529786c313 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:34.616883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ac654-481f-4e8c-8420-99529786c313 admin admin] PATH_INFO: `/v3/projects/8a61f14f37ec4a9eb97a1d069eb5ddcf/users/ee06ea0086014295849d3df1528dc2cd/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:34.619151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ac654-481f-4e8c-8420-99529786c313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a61f14f37ec4a9eb97a1d069eb5ddcf, user_id=ee06ea0086014295849d3df1528dc2cd, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:34.622207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ac654-481f-4e8c-8420-99529786c313 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:34.635412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1326/2651] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:34 2022] PUT /identity/v3/projects/8a61f14f37ec4a9eb97a1d069eb5ddcf/users/ee06ea0086014295849d3df1528dc2cd/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:34.647453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989838-1f6b-4f71-83ec-2666da9b1355 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:34.647745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989838-1f6b-4f71-83ec-2666da9b1355 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:34.648055 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989838-1f6b-4f71-83ec-2666da9b1355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:34.682957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-05989838-1f6b-4f71-83ec-2666da9b1355 None None] MFA Rules not processed for user `ee06ea0086014295849d3df1528dc2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:34.684217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-05989838-1f6b-4f71-83ec-2666da9b1355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:34.769924 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1326/2652] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:34.788185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ac654-481f-4e8c-8420-99529786c313 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:34.790059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-689ac654-481f-4e8c-8420-99529786c313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:34.857473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ac654-481f-4e8c-8420-99529786c313 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:34.861651 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f3b25-2145-4fd4-b02c-7e44d120823b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:34.862242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f3b25-2145-4fd4-b02c-7e44d120823b service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:34.869174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f3b25-2145-4fd4-b02c-7e44d120823b service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:34.869174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f3b25-2145-4fd4-b02c-7e44d120823b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:34.874894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f3b25-2145-4fd4-b02c-7e44d120823b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:34.876745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f3b25-2145-4fd4-b02c-7e44d120823b service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:34.888067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1327/2653] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:06:35.648584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05989838-1f6b-4f71-83ec-2666da9b1355 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:35.650647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-05989838-1f6b-4f71-83ec-2666da9b1355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:35.664987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05989838-1f6b-4f71-83ec-2666da9b1355 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:35.669024 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c70077-c5b3-4b3a-a9de-b2c2e3c0e75f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:35.669804 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c70077-c5b3-4b3a-a9de-b2c2e3c0e75f service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:35.670162 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c70077-c5b3-4b3a-a9de-b2c2e3c0e75f service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:35.670513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c70077-c5b3-4b3a-a9de-b2c2e3c0e75f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:35.692971 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:35.751967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c70077-c5b3-4b3a-a9de-b2c2e3c0e75f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:35.754799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c70077-c5b3-4b3a-a9de-b2c2e3c0e75f service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:35.767208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1327/2654] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:06:37.262565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f3b25-2145-4fd4-b02c-7e44d120823b service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:37.264972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2e4f3b25-2145-4fd4-b02c-7e44d120823b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:37.386385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:37.395462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f3b25-2145-4fd4-b02c-7e44d120823b service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:37.404272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21596e9e-d392-4969-938a-f3098ba0161d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:37.404982 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596e9e-d392-4969-938a-f3098ba0161d service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:37.405281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596e9e-d392-4969-938a-f3098ba0161d service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:37.405560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596e9e-d392-4969-938a-f3098ba0161d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:37.407628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-21596e9e-d392-4969-938a-f3098ba0161d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:37.426676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21596e9e-d392-4969-938a-f3098ba0161d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:37.428702 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21596e9e-d392-4969-938a-f3098ba0161d service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:37.442978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1328/2655] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:40.185825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c70077-c5b3-4b3a-a9de-b2c2e3c0e75f service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:40.196658 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c70077-c5b3-4b3a-a9de-b2c2e3c0e75f service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:40.201454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f2fb2-492b-4e51-969e-c813b670c93a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:40.202263 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f2fb2-492b-4e51-969e-c813b670c93a service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:40.202547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f2fb2-492b-4e51-969e-c813b670c93a service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:40.202822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f2fb2-492b-4e51-969e-c813b670c93a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:40.279519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f2fb2-492b-4e51-969e-c813b670c93a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:40.281087 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f2fb2-492b-4e51-969e-c813b670c93a service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:40.293507 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1328/2656] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:06:42.424476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21596e9e-d392-4969-938a-f3098ba0161d service nova] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:42.426475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-21596e9e-d392-4969-938a-f3098ba0161d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:42.442969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21596e9e-d392-4969-938a-f3098ba0161d service nova] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:42.451368 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379adfb8-9a39-42e2-8fcd-ec168e3808c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:42.452259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379adfb8-9a39-42e2-8fcd-ec168e3808c2 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:42.452895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379adfb8-9a39-42e2-8fcd-ec168e3808c2 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:42.453346 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379adfb8-9a39-42e2-8fcd-ec168e3808c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:42.456969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-379adfb8-9a39-42e2-8fcd-ec168e3808c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:42.476174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379adfb8-9a39-42e2-8fcd-ec168e3808c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:42.478388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379adfb8-9a39-42e2-8fcd-ec168e3808c2 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:42.500031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1329/2657] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:06:43.250399 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f2fb2-492b-4e51-969e-c813b670c93a service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:43.399445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f2fb2-492b-4e51-969e-c813b670c93a service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:43.406764 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:43.408740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:43.408996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:43.410046 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] PATH_INFO: `/v3/users/2921c88157614e52bd26d955b5469fc6` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:43.412022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2921c88157614e52bd26d955b5469fc6)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:43.416160 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:43.515556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2921c88157614e52bd26d955b5469fc6'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:43.528246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2921c88157614e52bd26d955b5469fc6'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:43.529681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2921c88157614e52bd26d955b5469fc6'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:43.539720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:43.559348 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2921c88157614e52bd26d955b5469fc6'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:43.574188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2921c88157614e52bd26d955b5469fc6'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:43.586952 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1329/2658] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:06:43 2022] DELETE /identity/v3/users/2921c88157614e52bd26d955b5469fc6 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:43.593749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93b2d-5d14-4580-b559-4fe3ed4d24a4 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:43.594625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93b2d-5d14-4580-b559-4fe3ed4d24a4 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:43.594625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93b2d-5d14-4580-b559-4fe3ed4d24a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:43.621926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-03f93b2d-5d14-4580-b559-4fe3ed4d24a4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:43.623307 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-03f93b2d-5d14-4580-b559-4fe3ed4d24a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:43.714823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1330/2659] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:43.728081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:43.730213 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:43.819793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce2e70-fe92-4872-bbe2-88e8a0cc6c01 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:43.823845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25c21d-e7bf-41ae-8d17-a670fe2c120c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:43.824778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25c21d-e7bf-41ae-8d17-a670fe2c120c service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:43.825227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25c21d-e7bf-41ae-8d17-a670fe2c120c service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:43.825519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25c21d-e7bf-41ae-8d17-a670fe2c120c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:43.837427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25c21d-e7bf-41ae-8d17-a670fe2c120c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:43.839256 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25c21d-e7bf-41ae-8d17-a670fe2c120c service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:43.853695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1330/2660] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:06:44.340675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f93b2d-5d14-4580-b559-4fe3ed4d24a4 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:44.342898 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-03f93b2d-5d14-4580-b559-4fe3ed4d24a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:44.370421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f93b2d-5d14-4580-b559-4fe3ed4d24a4 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:44.380060 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:44.381656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:44.382393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:44.382830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] PATH_INFO: `/v3/projects/bdef8f59891044dfb7baa8647b764efd` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:44.385277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdef8f59891044dfb7baa8647b764efd)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:44.389380 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:44.484807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdef8f59891044dfb7baa8647b764efd', 'request_id': 'req-cd524821-83ad-409b-8108-e7398bc60792'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:44.498537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdef8f59891044dfb7baa8647b764efd', 'request_id': 'req-cd524821-83ad-409b-8108-e7398bc60792'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:44.513443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdef8f59891044dfb7baa8647b764efd', 'request_id': 'req-cd524821-83ad-409b-8108-e7398bc60792'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:44.535074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] The token cache is being invalidate because project bdef8f59891044dfb7baa8647b764efd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:06:44.535583 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:44.536771 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:44.539499 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1331/2661] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:06:44 2022] DELETE /identity/v3/projects/bdef8f59891044dfb7baa8647b764efd => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:06:44.543932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25c21d-e7bf-41ae-8d17-a670fe2c120c service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:44.556625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25c21d-e7bf-41ae-8d17-a670fe2c120c service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:44.664737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:44.665708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:44.666034 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:44.666384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] PATH_INFO: `/v3/users/0376bdc93fff468ea7a3b5fc4087cce4` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:44.667921 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0376bdc93fff468ea7a3b5fc4087cce4)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:44.670193 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:44.736550 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0376bdc93fff468ea7a3b5fc4087cce4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:44.746919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0376bdc93fff468ea7a3b5fc4087cce4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:44.747699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0376bdc93fff468ea7a3b5fc4087cce4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:44.766156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0376bdc93fff468ea7a3b5fc4087cce4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:44.776502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0376bdc93fff468ea7a3b5fc4087cce4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:44.778601 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:44.789185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1331/2662] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:06:44 2022] DELETE /identity/v3/users/0376bdc93fff468ea7a3b5fc4087cce4 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:06:45.218458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:45.221773 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:45.249460 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd524821-83ad-409b-8108-e7398bc60792 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:45.329146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:45.330042 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:45.330390 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:45.330717 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] PATH_INFO: `/v3/projects/a78f3ba51c2b42c187eb1c7174ec3f13` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:45.332214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a78f3ba51c2b42c187eb1c7174ec3f13)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:45.334336 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:45.338013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:45.411637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a78f3ba51c2b42c187eb1c7174ec3f13', 'request_id': 'req-e502e8b9-b426-49cc-86a5-64c1f7bee627'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:45.422779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a78f3ba51c2b42c187eb1c7174ec3f13', 'request_id': 'req-e502e8b9-b426-49cc-86a5-64c1f7bee627'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:45.431531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a78f3ba51c2b42c187eb1c7174ec3f13', 'request_id': 'req-e502e8b9-b426-49cc-86a5-64c1f7bee627'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:45.445024 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] The token cache is being invalidate because project a78f3ba51c2b42c187eb1c7174ec3f13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:06:45.445366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:45.446098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:45.447893 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1332/2663] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:06:45 2022] DELETE /identity/v3/projects/a78f3ba51c2b42c187eb1c7174ec3f13 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:45.455539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:45.458063 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:45.477759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f26ec8-21c9-4404-bc71-1142bc9212e2 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:45.562987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2695e8-f196-4c9a-8b28-240027dd70f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:45.565347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2695e8-f196-4c9a-8b28-240027dd70f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:45.565849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2695e8-f196-4c9a-8b28-240027dd70f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:45.566340 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2695e8-f196-4c9a-8b28-240027dd70f0 admin admin] PATH_INFO: `/v3/users/ee06ea0086014295849d3df1528dc2cd` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:45.569051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2695e8-f196-4c9a-8b28-240027dd70f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee06ea0086014295849d3df1528dc2cd)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:45.573538 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2695e8-f196-4c9a-8b28-240027dd70f0 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:45.657874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-bb2695e8-f196-4c9a-8b28-240027dd70f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee06ea0086014295849d3df1528dc2cd'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:45.670518 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-bb2695e8-f196-4c9a-8b28-240027dd70f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee06ea0086014295849d3df1528dc2cd'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:45.671463 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-bb2695e8-f196-4c9a-8b28-240027dd70f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee06ea0086014295849d3df1528dc2cd'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:45.693717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-bb2695e8-f196-4c9a-8b28-240027dd70f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee06ea0086014295849d3df1528dc2cd'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:45.708382 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-bb2695e8-f196-4c9a-8b28-240027dd70f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee06ea0086014295849d3df1528dc2cd'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:45.720013 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1332/2664] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:06:45 2022] DELETE /identity/v3/users/ee06ea0086014295849d3df1528dc2cd => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:46.111894 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:46.113437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:46.127217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502e8b9-b426-49cc-86a5-64c1f7bee627 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:46.221235 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:46.222693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:46.223189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:46.223671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] PATH_INFO: `/v3/projects/8a61f14f37ec4a9eb97a1d069eb5ddcf` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:46.225680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a61f14f37ec4a9eb97a1d069eb5ddcf)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:46.228118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:46.294733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a61f14f37ec4a9eb97a1d069eb5ddcf', 'request_id': 'req-bf9f8e90-828a-46d8-86df-c8075683dcbf'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:46.307895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a61f14f37ec4a9eb97a1d069eb5ddcf', 'request_id': 'req-bf9f8e90-828a-46d8-86df-c8075683dcbf'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:46.319081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a61f14f37ec4a9eb97a1d069eb5ddcf', 'request_id': 'req-bf9f8e90-828a-46d8-86df-c8075683dcbf'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:46.339697 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] The token cache is being invalidate because project 8a61f14f37ec4a9eb97a1d069eb5ddcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:06:46.339995 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:46.340892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-bf9f8e90-828a-46d8-86df-c8075683dcbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:46.343237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1333/2665] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:06:46 2022] DELETE /identity/v3/projects/8a61f14f37ec4a9eb97a1d069eb5ddcf => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:46.353004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e220c-0584-4b46-ae5c-0b703815dc37 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:46.354104 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e220c-0584-4b46-ae5c-0b703815dc37 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:46.354104 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e220c-0584-4b46-ae5c-0b703815dc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:46.375714 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-947e220c-0584-4b46-ae5c-0b703815dc37 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:46.377071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-947e220c-0584-4b46-ae5c-0b703815dc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:46.488435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1333/2666] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:06:46.504357 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace23fd-a8e2-4381-82df-9f6d1020951d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:46.504914 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace23fd-a8e2-4381-82df-9f6d1020951d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:46.505507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace23fd-a8e2-4381-82df-9f6d1020951d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:46.542004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-dace23fd-a8e2-4381-82df-9f6d1020951d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:46.544142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dace23fd-a8e2-4381-82df-9f6d1020951d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:46.558844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1334/2667] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:46.563703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e220c-0584-4b46-ae5c-0b703815dc37 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:46.575701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e220c-0584-4b46-ae5c-0b703815dc37 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:46.580889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d77bb2-a4e6-42b1-8c00-ebc9b0a1664e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:46.581573 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d77bb2-a4e6-42b1-8c00-ebc9b0a1664e admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:46.581856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d77bb2-a4e6-42b1-8c00-ebc9b0a1664e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:46.582135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d77bb2-a4e6-42b1-8c00-ebc9b0a1664e admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:46.582876 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d77bb2-a4e6-42b1-8c00-ebc9b0a1664e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:06:46.583181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d77bb2-a4e6-42b1-8c00-ebc9b0a1664e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:46.584841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d77bb2-a4e6-42b1-8c00-ebc9b0a1664e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:46.599085 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1334/2668] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:06:46.603896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace23fd-a8e2-4381-82df-9f6d1020951d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:46.617223 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace23fd-a8e2-4381-82df-9f6d1020951d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:46.624476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc934e1-b29a-491e-b9d5-5c850badb319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:46.625220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc934e1-b29a-491e-b9d5-5c850badb319 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:46.625573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc934e1-b29a-491e-b9d5-5c850badb319 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:46.625922 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc934e1-b29a-491e-b9d5-5c850badb319 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:46.626719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc934e1-b29a-491e-b9d5-5c850badb319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:46.628812 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc934e1-b29a-491e-b9d5-5c850badb319 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:46.645339 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:46.662068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1335/2669] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:06:46 2022] POST /identity/v3/projects => generated 399 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:06:46.667344 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d77bb2-a4e6-42b1-8c00-ebc9b0a1664e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:46.675701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:46.680745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d77bb2-a4e6-42b1-8c00-ebc9b0a1664e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:46.752555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292d307-7460-49f5-a88a-45f12a0325bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:46.753121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292d307-7460-49f5-a88a-45f12a0325bd admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:46.753412 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292d307-7460-49f5-a88a-45f12a0325bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:46.753687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292d307-7460-49f5-a88a-45f12a0325bd admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:46.757893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292d307-7460-49f5-a88a-45f12a0325bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:46.757893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292d307-7460-49f5-a88a-45f12a0325bd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:46.779410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1335/2670] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:06:46 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:46.784363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc934e1-b29a-491e-b9d5-5c850badb319 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:46.796798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc934e1-b29a-491e-b9d5-5c850badb319 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:46.802724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26004b-870b-48dc-b556-68f8318dbabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:46.803411 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26004b-870b-48dc-b556-68f8318dbabd admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:46.803695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26004b-870b-48dc-b556-68f8318dbabd admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:46.803970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26004b-870b-48dc-b556-68f8318dbabd admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:46.805159 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26004b-870b-48dc-b556-68f8318dbabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:46.806641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26004b-870b-48dc-b556-68f8318dbabd admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:46.818761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1336/2671] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:46.823491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292d307-7460-49f5-a88a-45f12a0325bd admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:46.834349 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292d307-7460-49f5-a88a-45f12a0325bd admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:46.841881 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412809a8-866d-41df-95db-610395ff282f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:46.843041 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412809a8-866d-41df-95db-610395ff282f admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:46.843617 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412809a8-866d-41df-95db-610395ff282f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:46.844117 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412809a8-866d-41df-95db-610395ff282f admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:46.846333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412809a8-866d-41df-95db-610395ff282f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:46.849606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412809a8-866d-41df-95db-610395ff282f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:46.871360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1336/2672] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:46.875278 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26004b-870b-48dc-b556-68f8318dbabd admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:46.887157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26004b-870b-48dc-b556-68f8318dbabd admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:46.893214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c05d4-cf56-49d3-9dcd-aa4baf8164f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:46.894250 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c05d4-cf56-49d3-9dcd-aa4baf8164f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:46.894733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c05d4-cf56-49d3-9dcd-aa4baf8164f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:46.895230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c05d4-cf56-49d3-9dcd-aa4baf8164f6 admin admin] PATH_INFO: `/v3/projects/2c6dc2367a0444d6ab591e39de38907f/users/9970f2b44c5e4e7bb3cee98b51922e05/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:46.915667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c05d4-cf56-49d3-9dcd-aa4baf8164f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6dc2367a0444d6ab591e39de38907f, user_id=9970f2b44c5e4e7bb3cee98b51922e05, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:46.918672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c05d4-cf56-49d3-9dcd-aa4baf8164f6 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:46.937084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1337/2673] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:46 2022] PUT /identity/v3/projects/2c6dc2367a0444d6ab591e39de38907f/users/9970f2b44c5e4e7bb3cee98b51922e05/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:06:46.955645 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307a7a7-8f2c-4f13-b8be-14d97caf5239 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:46.956161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307a7a7-8f2c-4f13-b8be-14d97caf5239 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:46.956712 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307a7a7-8f2c-4f13-b8be-14d97caf5239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:47.007779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-9307a7a7-8f2c-4f13-b8be-14d97caf5239 None None] MFA Rules not processed for user `9970f2b44c5e4e7bb3cee98b51922e05`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:47.009922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9307a7a7-8f2c-4f13-b8be-14d97caf5239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:47.130177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1337/2674] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:47.137305 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c05d4-cf56-49d3-9dcd-aa4baf8164f6 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:47.221752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c05d4-cf56-49d3-9dcd-aa4baf8164f6 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:47.229660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c6072-eb77-42c3-bf46-150a59ca38f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:47.230761 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c6072-eb77-42c3-bf46-150a59ca38f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:47.231234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c6072-eb77-42c3-bf46-150a59ca38f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:47.231709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c6072-eb77-42c3-bf46-150a59ca38f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:47.233081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c6072-eb77-42c3-bf46-150a59ca38f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:47.237371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c6072-eb77-42c3-bf46-150a59ca38f2 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:47.284471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1338/2675] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:06:47 2022] POST /identity/v3/projects => generated 399 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:06:47.289772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307a7a7-8f2c-4f13-b8be-14d97caf5239 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:47.306625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307a7a7-8f2c-4f13-b8be-14d97caf5239 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:47.386733 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6c3ef-b16a-4e1c-931d-8105948b8f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:47.387451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c3ef-b16a-4e1c-931d-8105948b8f6c admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:47.387734 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c3ef-b16a-4e1c-931d-8105948b8f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:47.388018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c3ef-b16a-4e1c-931d-8105948b8f6c admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:47.388775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6c3ef-b16a-4e1c-931d-8105948b8f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:47.390726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6c3ef-b16a-4e1c-931d-8105948b8f6c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:47.415476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1338/2676] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:06:47 2022] POST /identity/v3/users => generated 306 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:06:47.421081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c6072-eb77-42c3-bf46-150a59ca38f2 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:47.433246 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c6072-eb77-42c3-bf46-150a59ca38f2 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:47.438471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9e9eb-de7c-4451-b5b9-f9a458cb42f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:47.439854 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9e9eb-de7c-4451-b5b9-f9a458cb42f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:47.440124 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9e9eb-de7c-4451-b5b9-f9a458cb42f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:47.440450 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9e9eb-de7c-4451-b5b9-f9a458cb42f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:47.441198 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9e9eb-de7c-4451-b5b9-f9a458cb42f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:47.442833 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9e9eb-de7c-4451-b5b9-f9a458cb42f8 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:47.455369 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1339/2677] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:47.461644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6c3ef-b16a-4e1c-931d-8105948b8f6c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:47.467655 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:47.475287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6c3ef-b16a-4e1c-931d-8105948b8f6c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:47.483165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447b4a6-e7f2-4330-9bdc-063be8a7be9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:47.485058 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447b4a6-e7f2-4330-9bdc-063be8a7be9f admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:47.485471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447b4a6-e7f2-4330-9bdc-063be8a7be9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:47.486204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447b4a6-e7f2-4330-9bdc-063be8a7be9f admin admin] PATH_INFO: `/v3/projects/75f950cf6ef8441599bdd2e6b6cb2227/users/a6216fd50cdd4dc1adc90a8ac9941e8c/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:47.523772 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447b4a6-e7f2-4330-9bdc-063be8a7be9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f950cf6ef8441599bdd2e6b6cb2227, user_id=a6216fd50cdd4dc1adc90a8ac9941e8c, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:47.528820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447b4a6-e7f2-4330-9bdc-063be8a7be9f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:47.550766 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1339/2678] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:47 2022] PUT /identity/v3/projects/75f950cf6ef8441599bdd2e6b6cb2227/users/a6216fd50cdd4dc1adc90a8ac9941e8c/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:47.555514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9e9eb-de7c-4451-b5b9-f9a458cb42f8 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:47.574781 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9e9eb-de7c-4451-b5b9-f9a458cb42f8 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:47.665406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01921330-56d4-4b7f-a139-d2b9e1a3e9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:47.666083 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01921330-56d4-4b7f-a139-d2b9e1a3e9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:47.666388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01921330-56d4-4b7f-a139-d2b9e1a3e9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:47.666723 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01921330-56d4-4b7f-a139-d2b9e1a3e9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:47.667531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01921330-56d4-4b7f-a139-d2b9e1a3e9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:47.669638 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01921330-56d4-4b7f-a139-d2b9e1a3e9b9 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:47.685916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1340/2679] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:47.691296 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447b4a6-e7f2-4330-9bdc-063be8a7be9f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:47.707564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447b4a6-e7f2-4330-9bdc-063be8a7be9f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:47.714278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743792a3-fdaa-40e0-9c24-5a6ff4451fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:47.715470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743792a3-fdaa-40e0-9c24-5a6ff4451fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:47.715934 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743792a3-fdaa-40e0-9c24-5a6ff4451fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:47.716364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743792a3-fdaa-40e0-9c24-5a6ff4451fbb admin admin] PATH_INFO: `/v3/projects/75f950cf6ef8441599bdd2e6b6cb2227/users/a6216fd50cdd4dc1adc90a8ac9941e8c/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:47.718963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743792a3-fdaa-40e0-9c24-5a6ff4451fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f950cf6ef8441599bdd2e6b6cb2227, user_id=a6216fd50cdd4dc1adc90a8ac9941e8c, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:47.722018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743792a3-fdaa-40e0-9c24-5a6ff4451fbb admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:47.736918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1340/2680] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:47 2022] PUT /identity/v3/projects/75f950cf6ef8441599bdd2e6b6cb2227/users/a6216fd50cdd4dc1adc90a8ac9941e8c/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:06:47.749709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245ca30-23f6-4600-b959-04234ee56eb2 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:47.749709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245ca30-23f6-4600-b959-04234ee56eb2 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:47.749709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245ca30-23f6-4600-b959-04234ee56eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:47.783059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-f245ca30-23f6-4600-b959-04234ee56eb2 None None] MFA Rules not processed for user `a6216fd50cdd4dc1adc90a8ac9941e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:47.784665 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f245ca30-23f6-4600-b959-04234ee56eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:47.816190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:47.895969 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1341/2681] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:06:47.908889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743792a3-fdaa-40e0-9c24-5a6ff4451fbb admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:47.910908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-743792a3-fdaa-40e0-9c24-5a6ff4451fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:47.998515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743792a3-fdaa-40e0-9c24-5a6ff4451fbb admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:48.006228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1afd1-ffa5-4a33-a2e1-9e22e3d1f2c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:48.006936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1afd1-ffa5-4a33-a2e1-9e22e3d1f2c2 service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:48.007440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1afd1-ffa5-4a33-a2e1-9e22e3d1f2c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:48.007984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1afd1-ffa5-4a33-a2e1-9e22e3d1f2c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:48.026295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1afd1-ffa5-4a33-a2e1-9e22e3d1f2c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:48.029217 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1afd1-ffa5-4a33-a2e1-9e22e3d1f2c2 service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:48.049213 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1341/2682] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:47 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:48.145763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245ca30-23f6-4600-b959-04234ee56eb2 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:48.157432 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245ca30-23f6-4600-b959-04234ee56eb2 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:48.164466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67cc3cd-e527-499e-80ba-2a6962abddd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:48.165221 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cc3cd-e527-499e-80ba-2a6962abddd3 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:48.165582 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cc3cd-e527-499e-80ba-2a6962abddd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:48.165941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cc3cd-e527-499e-80ba-2a6962abddd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:48.274799 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67cc3cd-e527-499e-80ba-2a6962abddd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:48.277477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67cc3cd-e527-499e-80ba-2a6962abddd3 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:48.290749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1342/2683] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:52.816859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1afd1-ffa5-4a33-a2e1-9e22e3d1f2c2 service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:52.905926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1afd1-ffa5-4a33-a2e1-9e22e3d1f2c2 service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:52.913485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:52.914914 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:52.915378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:52.915927 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] PATH_INFO: `/v3/users/9970f2b44c5e4e7bb3cee98b51922e05` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:52.918083 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9970f2b44c5e4e7bb3cee98b51922e05)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:52.921634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:53.001797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9970f2b44c5e4e7bb3cee98b51922e05'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:53.011423 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9970f2b44c5e4e7bb3cee98b51922e05'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:53.012171 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9970f2b44c5e4e7bb3cee98b51922e05'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:53.033499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:53.035909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9970f2b44c5e4e7bb3cee98b51922e05'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:53.050539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9970f2b44c5e4e7bb3cee98b51922e05'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:53.059072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1342/2684] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:06:52 2022] DELETE /identity/v3/users/9970f2b44c5e4e7bb3cee98b51922e05 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:53.064414 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0cea-6e23-4a7f-9d30-1d060b1c9c1b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:53.065436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0cea-6e23-4a7f-9d30-1d060b1c9c1b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:53.065436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c0cea-6e23-4a7f-9d30-1d060b1c9c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:53.087787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-770c0cea-6e23-4a7f-9d30-1d060b1c9c1b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:53.089704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-770c0cea-6e23-4a7f-9d30-1d060b1c9c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:53.169535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1343/2685] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:53.183693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:53.185561 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:53.299458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5066f-d5c8-48d0-847c-db542cc3f4cf admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:53.306499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308879d-f84f-4db3-a603-8e4b705ef22e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:53.307585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308879d-f84f-4db3-a603-8e4b705ef22e service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:53.308081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308879d-f84f-4db3-a603-8e4b705ef22e service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:53.308615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308879d-f84f-4db3-a603-8e4b705ef22e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:53.327489 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308879d-f84f-4db3-a603-8e4b705ef22e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:53.330377 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308879d-f84f-4db3-a603-8e4b705ef22e service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:53.351095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1343/2686] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:53.760137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c0cea-6e23-4a7f-9d30-1d060b1c9c1b None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:53.762445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-770c0cea-6e23-4a7f-9d30-1d060b1c9c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:53.783684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c0cea-6e23-4a7f-9d30-1d060b1c9c1b None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:53.804423 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:53.806442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:53.806925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:53.807409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] PATH_INFO: `/v3/projects/2c6dc2367a0444d6ab591e39de38907f` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:53.810459 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6dc2367a0444d6ab591e39de38907f)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:53.815798 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:53.891612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:53.894094 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6dc2367a0444d6ab591e39de38907f', 'request_id': 'req-fc22f35e-438b-4b88-973a-7b6901870c1e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:53.904054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6dc2367a0444d6ab591e39de38907f', 'request_id': 'req-fc22f35e-438b-4b88-973a-7b6901870c1e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:53.912359 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6dc2367a0444d6ab591e39de38907f', 'request_id': 'req-fc22f35e-438b-4b88-973a-7b6901870c1e'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:53.926029 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] The token cache is being invalidate because project 2c6dc2367a0444d6ab591e39de38907f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:06:53.926377 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:53.927139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:53.928926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1344/2687] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:06:53 2022] DELETE /identity/v3/projects/2c6dc2367a0444d6ab591e39de38907f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:53.934532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308879d-f84f-4db3-a603-8e4b705ef22e service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:53.947174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308879d-f84f-4db3-a603-8e4b705ef22e service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:54.023620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5b60d-0842-4a9d-8d54-b91e5bc37ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:54.024739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5b60d-0842-4a9d-8d54-b91e5bc37ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:54.025081 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5b60d-0842-4a9d-8d54-b91e5bc37ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:54.025429 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5b60d-0842-4a9d-8d54-b91e5bc37ac7 admin admin] PATH_INFO: `/v3/users/a6216fd50cdd4dc1adc90a8ac9941e8c` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:54.026860 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5b60d-0842-4a9d-8d54-b91e5bc37ac7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6216fd50cdd4dc1adc90a8ac9941e8c)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:54.029185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5b60d-0842-4a9d-8d54-b91e5bc37ac7 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:54.100786 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-aca5b60d-0842-4a9d-8d54-b91e5bc37ac7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6216fd50cdd4dc1adc90a8ac9941e8c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:54.113984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-aca5b60d-0842-4a9d-8d54-b91e5bc37ac7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6216fd50cdd4dc1adc90a8ac9941e8c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:54.115191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-aca5b60d-0842-4a9d-8d54-b91e5bc37ac7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6216fd50cdd4dc1adc90a8ac9941e8c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:54.132226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-aca5b60d-0842-4a9d-8d54-b91e5bc37ac7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6216fd50cdd4dc1adc90a8ac9941e8c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:54.140867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-aca5b60d-0842-4a9d-8d54-b91e5bc37ac7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6216fd50cdd4dc1adc90a8ac9941e8c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:54.154051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1344/2688] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:06:53 2022] DELETE /identity/v3/users/a6216fd50cdd4dc1adc90a8ac9941e8c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:54.641459 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:54.644376 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:54.661365 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22f35e-438b-4b88-973a-7b6901870c1e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:54.776404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:54.778866 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:54.779519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:54.780138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] PATH_INFO: `/v3/projects/75f950cf6ef8441599bdd2e6b6cb2227` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:54.784162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f950cf6ef8441599bdd2e6b6cb2227)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:54.787567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:54.873340 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f950cf6ef8441599bdd2e6b6cb2227', 'request_id': 'req-b18862a0-25fa-4c0f-ac26-51eacba882aa'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:54.888189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f950cf6ef8441599bdd2e6b6cb2227', 'request_id': 'req-b18862a0-25fa-4c0f-ac26-51eacba882aa'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:54.902240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f950cf6ef8441599bdd2e6b6cb2227', 'request_id': 'req-b18862a0-25fa-4c0f-ac26-51eacba882aa'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:54.921028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] The token cache is being invalidate because project 75f950cf6ef8441599bdd2e6b6cb2227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:06:54.921518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:54.922605 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b18862a0-25fa-4c0f-ac26-51eacba882aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:54.925837 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1345/2689] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:06:54 2022] DELETE /identity/v3/projects/75f950cf6ef8441599bdd2e6b6cb2227 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:54.931500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b06d6-f79a-4de7-a190-18fad4b998b2 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:54.931981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b06d6-f79a-4de7-a190-18fad4b998b2 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:54.932485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b06d6-f79a-4de7-a190-18fad4b998b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:54.974218 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0d4b06d6-f79a-4de7-a190-18fad4b998b2 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:54.976123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0d4b06d6-f79a-4de7-a190-18fad4b998b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:54.983298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:55.087272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1345/2690] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:55.096862 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea5e62-abbc-47c1-9742-7ee80ad4c46d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:55.097205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea5e62-abbc-47c1-9742-7ee80ad4c46d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:55.097425 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea5e62-abbc-47c1-9742-7ee80ad4c46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:55.121165 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-b6ea5e62-abbc-47c1-9742-7ee80ad4c46d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:55.122473 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b6ea5e62-abbc-47c1-9742-7ee80ad4c46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:55.131558 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1346/2691] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:55.136837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b06d6-f79a-4de7-a190-18fad4b998b2 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:55.155801 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b06d6-f79a-4de7-a190-18fad4b998b2 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:55.164435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a191af-26c1-4e1d-9dbb-e37906dec96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:55.165687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a191af-26c1-4e1d-9dbb-e37906dec96f admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:55.166209 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a191af-26c1-4e1d-9dbb-e37906dec96f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:55.166718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a191af-26c1-4e1d-9dbb-e37906dec96f admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:55.168122 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a191af-26c1-4e1d-9dbb-e37906dec96f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:06:55.168725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a191af-26c1-4e1d-9dbb-e37906dec96f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:55.171963 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a191af-26c1-4e1d-9dbb-e37906dec96f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:55.197049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1346/2692] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:06:55.202915 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea5e62-abbc-47c1-9742-7ee80ad4c46d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:55.215238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea5e62-abbc-47c1-9742-7ee80ad4c46d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:55.221606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eea55e-aa2a-4bc7-bcad-b06cfcd8d24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:55.222301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eea55e-aa2a-4bc7-bcad-b06cfcd8d24f admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:55.222862 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eea55e-aa2a-4bc7-bcad-b06cfcd8d24f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:55.222862 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eea55e-aa2a-4bc7-bcad-b06cfcd8d24f admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:55.223669 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eea55e-aa2a-4bc7-bcad-b06cfcd8d24f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:55.225733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eea55e-aa2a-4bc7-bcad-b06cfcd8d24f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:55.258722 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1347/2693] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:06:55 2022] POST /identity/v3/projects => generated 361 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:06:55.263579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a191af-26c1-4e1d-9dbb-e37906dec96f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:55.279056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a191af-26c1-4e1d-9dbb-e37906dec96f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:55.365668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afb285-c156-474e-8521-ab5b8a5be9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:55.366596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afb285-c156-474e-8521-ab5b8a5be9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:55.366891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afb285-c156-474e-8521-ab5b8a5be9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:55.367170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afb285-c156-474e-8521-ab5b8a5be9c4 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:55.367928 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94afb285-c156-474e-8521-ab5b8a5be9c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:55.369916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94afb285-c156-474e-8521-ab5b8a5be9c4 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:55.398432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1347/2694] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:06:55 2022] POST /identity/v3/users => generated 281 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:06:55.403590 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eea55e-aa2a-4bc7-bcad-b06cfcd8d24f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:55.415567 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eea55e-aa2a-4bc7-bcad-b06cfcd8d24f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:55.420785 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bceda-b5b3-455c-bff6-02d2d0eb73c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:55.421472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bceda-b5b3-455c-bff6-02d2d0eb73c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:55.421779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bceda-b5b3-455c-bff6-02d2d0eb73c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:55.422113 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bceda-b5b3-455c-bff6-02d2d0eb73c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:55.422870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bceda-b5b3-455c-bff6-02d2d0eb73c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:55.425565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bceda-b5b3-455c-bff6-02d2d0eb73c6 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:55.439535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1348/2695] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:55.444023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afb285-c156-474e-8521-ab5b8a5be9c4 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:55.455845 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94afb285-c156-474e-8521-ab5b8a5be9c4 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:55.461562 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ee91b-b74d-4c9e-86e6-4ce62d4a1274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:55.462416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ee91b-b74d-4c9e-86e6-4ce62d4a1274 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:55.462763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ee91b-b74d-4c9e-86e6-4ce62d4a1274 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:55.463082 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ee91b-b74d-4c9e-86e6-4ce62d4a1274 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:55.463870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ee91b-b74d-4c9e-86e6-4ce62d4a1274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:55.465631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ee91b-b74d-4c9e-86e6-4ce62d4a1274 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:55.477959 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1348/2696] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:55.484830 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bceda-b5b3-455c-bff6-02d2d0eb73c6 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:55.496749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bceda-b5b3-455c-bff6-02d2d0eb73c6 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:55.502157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c882320-8271-4d23-9534-711d4ab77c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:55.503802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c882320-8271-4d23-9534-711d4ab77c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:55.504220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c882320-8271-4d23-9534-711d4ab77c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:55.504636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c882320-8271-4d23-9534-711d4ab77c08 admin admin] PATH_INFO: `/v3/projects/ccf8ed7463044397884409510de58b1d/users/564d5dc97de349188717d694aa5a8c14/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:55.526249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c882320-8271-4d23-9534-711d4ab77c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf8ed7463044397884409510de58b1d, user_id=564d5dc97de349188717d694aa5a8c14, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:55.530314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c882320-8271-4d23-9534-711d4ab77c08 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:55.550682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1349/2697] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:55 2022] PUT /identity/v3/projects/ccf8ed7463044397884409510de58b1d/users/564d5dc97de349188717d694aa5a8c14/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:06:55.564555 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5213710-1aad-4db8-846e-c4375767cebb None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:55.564872 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5213710-1aad-4db8-846e-c4375767cebb None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:55.565205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5213710-1aad-4db8-846e-c4375767cebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:55.598559 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-b5213710-1aad-4db8-846e-c4375767cebb None None] MFA Rules not processed for user `564d5dc97de349188717d694aa5a8c14`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:55.600046 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b5213710-1aad-4db8-846e-c4375767cebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:55.706016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1349/2698] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:06:55.743371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c882320-8271-4d23-9534-711d4ab77c08 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:55.745704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7c882320-8271-4d23-9534-711d4ab77c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:55.752775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:55.837177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c882320-8271-4d23-9534-711d4ab77c08 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:55.841762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41db0ec-5319-4638-a9c5-fe3c18f884cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:55.842448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41db0ec-5319-4638-a9c5-fe3c18f884cb service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:55.842724 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41db0ec-5319-4638-a9c5-fe3c18f884cb service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:55.843000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41db0ec-5319-4638-a9c5-fe3c18f884cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:55.854131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41db0ec-5319-4638-a9c5-fe3c18f884cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:55.855915 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41db0ec-5319-4638-a9c5-fe3c18f884cb service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:55.868663 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1350/2699] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:55.888205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5213710-1aad-4db8-846e-c4375767cebb None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:55.895106 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:55.981467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5213710-1aad-4db8-846e-c4375767cebb None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:55.987519 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:55.988973 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:55.989418 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:55.989916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] PATH_INFO: `/v3/users/564d5dc97de349188717d694aa5a8c14` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:55.991793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] RBAC: Authorizing `identity:delete_user(user_id=564d5dc97de349188717d694aa5a8c14)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:55.994726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:56.071323 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564d5dc97de349188717d694aa5a8c14'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:56.085050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564d5dc97de349188717d694aa5a8c14'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:56.086174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564d5dc97de349188717d694aa5a8c14'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:56.101746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564d5dc97de349188717d694aa5a8c14'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:56.110284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564d5dc97de349188717d694aa5a8c14'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:56.119167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1350/2700] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:06:55 2022] DELETE /identity/v3/users/564d5dc97de349188717d694aa5a8c14 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:56.126504 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eef3d8-3090-4298-8226-3cfcb63d2b5a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:56.126944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eef3d8-3090-4298-8226-3cfcb63d2b5a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:56.127846 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eef3d8-3090-4298-8226-3cfcb63d2b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:56.154888 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-d6eef3d8-3090-4298-8226-3cfcb63d2b5a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:56.157190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d6eef3d8-3090-4298-8226-3cfcb63d2b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:56.249531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1351/2701] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:56.263530 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:56.265402 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:56.343827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbaf570-6999-472a-954c-cca57bd0210d admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:56.347906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa3acf-8717-4700-a58c-3c02d533bab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:56.348642 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa3acf-8717-4700-a58c-3c02d533bab6 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:56.348932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa3acf-8717-4700-a58c-3c02d533bab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:56.349211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa3acf-8717-4700-a58c-3c02d533bab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:56.360616 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa3acf-8717-4700-a58c-3c02d533bab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:56.362468 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa3acf-8717-4700-a58c-3c02d533bab6 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:56.373891 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1351/2702] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:06:56.796530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eef3d8-3090-4298-8226-3cfcb63d2b5a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:56.799746 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d6eef3d8-3090-4298-8226-3cfcb63d2b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:56.819744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eef3d8-3090-4298-8226-3cfcb63d2b5a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:56.825681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:56.826559 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:56.826840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:56.827259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] PATH_INFO: `/v3/projects/ccf8ed7463044397884409510de58b1d` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:56.828924 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf8ed7463044397884409510de58b1d)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:56.831729 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:56.922177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf8ed7463044397884409510de58b1d', 'request_id': 'req-4ddbfdb3-cb26-44af-b721-3520b98ee72a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:56.935523 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf8ed7463044397884409510de58b1d', 'request_id': 'req-4ddbfdb3-cb26-44af-b721-3520b98ee72a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:56.945608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf8ed7463044397884409510de58b1d', 'request_id': 'req-4ddbfdb3-cb26-44af-b721-3520b98ee72a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:56.961751 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] The token cache is being invalidate because project ccf8ed7463044397884409510de58b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:06:56.962056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:56.962981 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-4ddbfdb3-cb26-44af-b721-3520b98ee72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:06:56.965754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1352/2703] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:06:56 2022] DELETE /identity/v3/projects/ccf8ed7463044397884409510de58b1d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:56.973261 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55036ba-3e06-45ea-b981-d0862b0473e1 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:56.973765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55036ba-3e06-45ea-b981-d0862b0473e1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:56.974208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55036ba-3e06-45ea-b981-d0862b0473e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:57.002666 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-c55036ba-3e06-45ea-b981-d0862b0473e1 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:57.004450 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c55036ba-3e06-45ea-b981-d0862b0473e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:57.039570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:57.105611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1352/2704] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:06:57.117211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77b5c5-ef4b-4b17-b0a9-74c82a1b99c3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:57.117639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77b5c5-ef4b-4b17-b0a9-74c82a1b99c3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:57.118127 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77b5c5-ef4b-4b17-b0a9-74c82a1b99c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:57.150535 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4e77b5c5-ef4b-4b17-b0a9-74c82a1b99c3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:57.153035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4e77b5c5-ef4b-4b17-b0a9-74c82a1b99c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:57.167887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1353/2705] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:57.173809 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55036ba-3e06-45ea-b981-d0862b0473e1 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:57.193322 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55036ba-3e06-45ea-b981-d0862b0473e1 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:57.201708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fcc57-29d5-4a8d-821a-f4903933c08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:57.202859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fcc57-29d5-4a8d-821a-f4903933c08e admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:57.203294 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fcc57-29d5-4a8d-821a-f4903933c08e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:57.203708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fcc57-29d5-4a8d-821a-f4903933c08e admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:57.206168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fcc57-29d5-4a8d-821a-f4903933c08e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:06:57.206168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fcc57-29d5-4a8d-821a-f4903933c08e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:57.208067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fcc57-29d5-4a8d-821a-f4903933c08e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:57.232156 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1353/2706] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:06:57.239212 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77b5c5-ef4b-4b17-b0a9-74c82a1b99c3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:57.250755 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77b5c5-ef4b-4b17-b0a9-74c82a1b99c3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:57.257211 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96709a8-e408-4358-870d-4894f54500ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:57.258224 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96709a8-e408-4358-870d-4894f54500ba admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:57.258631 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96709a8-e408-4358-870d-4894f54500ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:57.259068 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96709a8-e408-4358-870d-4894f54500ba admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:57.259821 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96709a8-e408-4358-870d-4894f54500ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:57.262471 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96709a8-e408-4358-870d-4894f54500ba admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:57.299328 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:06:57.312295 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1354/2707] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:06:57 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:06:57.318556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fcc57-29d5-4a8d-821a-f4903933c08e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:57.333665 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fcc57-29d5-4a8d-821a-f4903933c08e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:57.437204 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8e728-d4f8-4098-a522-886b97c884f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:57.438324 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8e728-d4f8-4098-a522-886b97c884f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:57.438759 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8e728-d4f8-4098-a522-886b97c884f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:57.439287 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8e728-d4f8-4098-a522-886b97c884f9 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:57.440597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8e728-d4f8-4098-a522-886b97c884f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:57.444071 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8e728-d4f8-4098-a522-886b97c884f9 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:57.474499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1354/2708] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:06:57 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:06:57.480589 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96709a8-e408-4358-870d-4894f54500ba admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:57.497066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96709a8-e408-4358-870d-4894f54500ba admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:57.504892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b4a9b-0312-4e18-a332-bc05856c19f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:57.505946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b4a9b-0312-4e18-a332-bc05856c19f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:57.506337 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b4a9b-0312-4e18-a332-bc05856c19f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:57.506972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b4a9b-0312-4e18-a332-bc05856c19f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:57.507770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b4a9b-0312-4e18-a332-bc05856c19f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:57.509714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b4a9b-0312-4e18-a332-bc05856c19f9 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:57.523923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1355/2709] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:06:57.529443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8e728-d4f8-4098-a522-886b97c884f9 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:57.541236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8e728-d4f8-4098-a522-886b97c884f9 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:57.546834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0192f2-fc94-4ca7-a1ca-7af667526c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:57.547589 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0192f2-fc94-4ca7-a1ca-7af667526c3d admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:57.547922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0192f2-fc94-4ca7-a1ca-7af667526c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:57.548839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0192f2-fc94-4ca7-a1ca-7af667526c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:57.549838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0192f2-fc94-4ca7-a1ca-7af667526c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:57.551794 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0192f2-fc94-4ca7-a1ca-7af667526c3d admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:57.565761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1355/2710] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:06:57.572012 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b4a9b-0312-4e18-a332-bc05856c19f9 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:57.585302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b4a9b-0312-4e18-a332-bc05856c19f9 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:57.590902 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a3464-f2fa-479c-8482-f3d338d4a548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:57.592102 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a3464-f2fa-479c-8482-f3d338d4a548 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:57.592531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a3464-f2fa-479c-8482-f3d338d4a548 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:57.592911 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a3464-f2fa-479c-8482-f3d338d4a548 admin admin] PATH_INFO: `/v3/projects/ed9b45eb7b1c4cc6bd56774371fe659b/users/84c4b81b6ac9481caa59b2caabae8a0a/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:57.618033 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a3464-f2fa-479c-8482-f3d338d4a548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9b45eb7b1c4cc6bd56774371fe659b, user_id=84c4b81b6ac9481caa59b2caabae8a0a, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:57.621354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a3464-f2fa-479c-8482-f3d338d4a548 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:57.642360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1356/2711] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:06:57 2022] PUT /identity/v3/projects/ed9b45eb7b1c4cc6bd56774371fe659b/users/84c4b81b6ac9481caa59b2caabae8a0a/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:06:57.657691 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceae386-fa96-4f88-9f7a-83d746391590 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:57.657691 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceae386-fa96-4f88-9f7a-83d746391590 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:57.657691 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceae386-fa96-4f88-9f7a-83d746391590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:57.711919 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-1ceae386-fa96-4f88-9f7a-83d746391590 None None] MFA Rules not processed for user `84c4b81b6ac9481caa59b2caabae8a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:06:57.714144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1ceae386-fa96-4f88-9f7a-83d746391590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:57.835956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1356/2712] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:06:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:06:57.852021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a3464-f2fa-479c-8482-f3d338d4a548 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:06:57.854947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ba0a3464-f2fa-479c-8482-f3d338d4a548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:06:57.963727 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a3464-f2fa-479c-8482-f3d338d4a548 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:06:57.976649 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae945b00-7c8d-49f8-87ca-874bb3a26102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:06:57.977292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae945b00-7c8d-49f8-87ca-874bb3a26102 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:06:57.977758 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae945b00-7c8d-49f8-87ca-874bb3a26102 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:06:57.978226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae945b00-7c8d-49f8-87ca-874bb3a26102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:06:57.993482 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae945b00-7c8d-49f8-87ca-874bb3a26102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:06:57.996314 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae945b00-7c8d-49f8-87ca-874bb3a26102 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:06:58.014202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1357/2713] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:06:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:07:07.830205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:08.006259 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:12.102582 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceae386-fa96-4f88-9f7a-83d746391590 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:12.105356 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1ceae386-fa96-4f88-9f7a-83d746391590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:12.288451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceae386-fa96-4f88-9f7a-83d746391590 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:12.297738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b4698-b5c0-4aaa-a91c-1f85766754e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:12.299738 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b4698-b5c0-4aaa-a91c-1f85766754e2 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:12.300211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b4698-b5c0-4aaa-a91c-1f85766754e2 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:12.300695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b4698-b5c0-4aaa-a91c-1f85766754e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:12.307544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9c3b4698-b5c0-4aaa-a91c-1f85766754e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:12.336064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b4698-b5c0-4aaa-a91c-1f85766754e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:12.339048 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b4698-b5c0-4aaa-a91c-1f85766754e2 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:12.357221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1357/2714] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:07:22.349091 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=34 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:23.692170 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae945b00-7c8d-49f8-87ca-874bb3a26102 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:23.791186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae945b00-7c8d-49f8-87ca-874bb3a26102 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:23.796881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:23.798256 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:23.798851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:23.799157 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] PATH_INFO: `/v3/users/84c4b81b6ac9481caa59b2caabae8a0a` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:23.801040 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c4b81b6ac9481caa59b2caabae8a0a)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:23.803093 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:23.883585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c4b81b6ac9481caa59b2caabae8a0a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:23.913984 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c4b81b6ac9481caa59b2caabae8a0a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:23.931163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c4b81b6ac9481caa59b2caabae8a0a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:23.932507 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c4b81b6ac9481caa59b2caabae8a0a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:23.958609 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c4b81b6ac9481caa59b2caabae8a0a'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:23.966986 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1358/2715] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:07:23 2022] DELETE /identity/v3/users/84c4b81b6ac9481caa59b2caabae8a0a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:07:23.973577 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffee498-ade9-4968-b721-0ba38621bec9 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:23.974387 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffee498-ade9-4968-b721-0ba38621bec9 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:23.974863 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffee498-ade9-4968-b721-0ba38621bec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:24.010132 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-1ffee498-ade9-4968-b721-0ba38621bec9 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:24.012399 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1ffee498-ade9-4968-b721-0ba38621bec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:24.142181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1358/2716] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:07:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:07:24.158841 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:24.162549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:24.274811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23b552-3f05-426c-bb2f-cde97de7f4e0 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:24.281421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c77461-bf01-4393-a354-3e4bf84f8814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:24.283131 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c77461-bf01-4393-a354-3e4bf84f8814 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:24.283635 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c77461-bf01-4393-a354-3e4bf84f8814 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:24.284145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c77461-bf01-4393-a354-3e4bf84f8814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:24.302695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c77461-bf01-4393-a354-3e4bf84f8814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:24.305586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c77461-bf01-4393-a354-3e4bf84f8814 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:24.325765 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1359/2717] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:07:24.771889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffee498-ade9-4968-b721-0ba38621bec9 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:24.784517 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-1ffee498-ade9-4968-b721-0ba38621bec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:24.810903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffee498-ade9-4968-b721-0ba38621bec9 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:24.823361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:24.825023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:24.825539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:24.825976 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] PATH_INFO: `/v3/projects/ed9b45eb7b1c4cc6bd56774371fe659b` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:24.828492 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed9b45eb7b1c4cc6bd56774371fe659b)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:24.832367 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:24.912553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed9b45eb7b1c4cc6bd56774371fe659b', 'request_id': 'req-41c20927-f269-4069-8a00-851d88f639c3'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:24.924009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed9b45eb7b1c4cc6bd56774371fe659b', 'request_id': 'req-41c20927-f269-4069-8a00-851d88f639c3'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:24.931603 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed9b45eb7b1c4cc6bd56774371fe659b', 'request_id': 'req-41c20927-f269-4069-8a00-851d88f639c3'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:24.948161 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] The token cache is being invalidate because project ed9b45eb7b1c4cc6bd56774371fe659b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:07:24.948553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:24.949311 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-41c20927-f269-4069-8a00-851d88f639c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:24.951289 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1359/2718] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:07:24 2022] DELETE /identity/v3/projects/ed9b45eb7b1c4cc6bd56774371fe659b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:07:24.958679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a9346-fc4b-42e0-a5db-f42657e5dabf None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:24.959620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a9346-fc4b-42e0-a5db-f42657e5dabf None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:24.959620 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a9346-fc4b-42e0-a5db-f42657e5dabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:24.993403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-475a9346-fc4b-42e0-a5db-f42657e5dabf None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:24.995645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-475a9346-fc4b-42e0-a5db-f42657e5dabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:25.025805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:25.118070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1360/2719] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:07:25.127836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a5d0-33d4-4ec0-b985-faeaad61dd1d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:25.128699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a5d0-33d4-4ec0-b985-faeaad61dd1d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:25.128907 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a5d0-33d4-4ec0-b985-faeaad61dd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:25.164754 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e731a5d0-33d4-4ec0-b985-faeaad61dd1d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:25.166062 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e731a5d0-33d4-4ec0-b985-faeaad61dd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:25.176841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1360/2720] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:07:25.182264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a9346-fc4b-42e0-a5db-f42657e5dabf None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:25.196366 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a9346-fc4b-42e0-a5db-f42657e5dabf None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:25.202541 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206532e-3b0f-4a3f-9267-79869cd16729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:25.203340 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206532e-3b0f-4a3f-9267-79869cd16729 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:25.203741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206532e-3b0f-4a3f-9267-79869cd16729 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:25.204681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206532e-3b0f-4a3f-9267-79869cd16729 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:25.205925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206532e-3b0f-4a3f-9267-79869cd16729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:07:25.206457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206532e-3b0f-4a3f-9267-79869cd16729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:25.208516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206532e-3b0f-4a3f-9267-79869cd16729 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:25.230790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1361/2721] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:07:25.238309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a5d0-33d4-4ec0-b985-faeaad61dd1d None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:25.254000 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a5d0-33d4-4ec0-b985-faeaad61dd1d None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:25.260922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf20f8-15a7-424e-935a-b4df629761c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:25.261607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf20f8-15a7-424e-935a-b4df629761c0 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:25.261893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf20f8-15a7-424e-935a-b4df629761c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:25.262175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf20f8-15a7-424e-935a-b4df629761c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:25.263182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf20f8-15a7-424e-935a-b4df629761c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:25.266054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf20f8-15a7-424e-935a-b4df629761c0 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:25.314253 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1361/2722] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:07:25 2022] POST /identity/v3/projects => generated 399 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:07:25.321531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206532e-3b0f-4a3f-9267-79869cd16729 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:25.341684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206532e-3b0f-4a3f-9267-79869cd16729 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:25.452213 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243194cb-d4c8-4a47-9a5b-4c185397f891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:25.453543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243194cb-d4c8-4a47-9a5b-4c185397f891 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:25.454115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243194cb-d4c8-4a47-9a5b-4c185397f891 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:25.454575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243194cb-d4c8-4a47-9a5b-4c185397f891 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:25.455959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243194cb-d4c8-4a47-9a5b-4c185397f891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:25.459903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243194cb-d4c8-4a47-9a5b-4c185397f891 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:25.499338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1362/2723] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:07:25 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:07:25.502028 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf20f8-15a7-424e-935a-b4df629761c0 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:25.521195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf20f8-15a7-424e-935a-b4df629761c0 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:25.530864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18cb38-bc9f-49e8-8c8c-2c1cfbd53d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:25.531702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18cb38-bc9f-49e8-8c8c-2c1cfbd53d34 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:25.532615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18cb38-bc9f-49e8-8c8c-2c1cfbd53d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:25.532915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18cb38-bc9f-49e8-8c8c-2c1cfbd53d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:25.533708 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18cb38-bc9f-49e8-8c8c-2c1cfbd53d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:25.535444 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18cb38-bc9f-49e8-8c8c-2c1cfbd53d34 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:25.550050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1362/2724] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:07:25.554891 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243194cb-d4c8-4a47-9a5b-4c185397f891 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:25.582331 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243194cb-d4c8-4a47-9a5b-4c185397f891 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:25.592941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06abf4-e337-4852-819b-be97405dc65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:25.594117 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06abf4-e337-4852-819b-be97405dc65f admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:25.595491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06abf4-e337-4852-819b-be97405dc65f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:25.595997 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06abf4-e337-4852-819b-be97405dc65f admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:25.597478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06abf4-e337-4852-819b-be97405dc65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:25.600757 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06abf4-e337-4852-819b-be97405dc65f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:25.624984 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1363/2725] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:07:25.631153 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18cb38-bc9f-49e8-8c8c-2c1cfbd53d34 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:25.645216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18cb38-bc9f-49e8-8c8c-2c1cfbd53d34 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:25.650445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e7103-2101-4447-83f5-ace1e831945f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:25.651373 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e7103-2101-4447-83f5-ace1e831945f admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:25.651654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e7103-2101-4447-83f5-ace1e831945f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:25.651931 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e7103-2101-4447-83f5-ace1e831945f admin admin] PATH_INFO: `/v3/projects/cd707eb241b64a05b637087f6a4870b4/users/372fdaca2fef4a45bb675451a08ac49f/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:25.683180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e7103-2101-4447-83f5-ace1e831945f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd707eb241b64a05b637087f6a4870b4, user_id=372fdaca2fef4a45bb675451a08ac49f, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:25.687020 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e7103-2101-4447-83f5-ace1e831945f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:25.716539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1363/2726] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:07:25 2022] PUT /identity/v3/projects/cd707eb241b64a05b637087f6a4870b4/users/372fdaca2fef4a45bb675451a08ac49f/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:07:25.736965 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d50e27-4bbc-445f-a78d-7d675b0a1b01 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:25.737464 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d50e27-4bbc-445f-a78d-7d675b0a1b01 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:25.738133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d50e27-4bbc-445f-a78d-7d675b0a1b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:25.796062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-61d50e27-4bbc-445f-a78d-7d675b0a1b01 None None] MFA Rules not processed for user `372fdaca2fef4a45bb675451a08ac49f`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:25.798067 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-61d50e27-4bbc-445f-a78d-7d675b0a1b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:25.904479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1364/2727] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:07:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:07:25.917656 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e7103-2101-4447-83f5-ace1e831945f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:25.919639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-be4e7103-2101-4447-83f5-ace1e831945f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:25.951019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:26.025455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e7103-2101-4447-83f5-ace1e831945f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:26.033330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5d4c3-0288-4faa-ba5c-f8a75a4d89bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:26.034042 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5d4c3-0288-4faa-ba5c-f8a75a4d89bc service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:26.034326 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5d4c3-0288-4faa-ba5c-f8a75a4d89bc service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:26.034606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5d4c3-0288-4faa-ba5c-f8a75a4d89bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:26.047859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5d4c3-0288-4faa-ba5c-f8a75a4d89bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:26.050314 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5d4c3-0288-4faa-ba5c-f8a75a4d89bc service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:26.065214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1364/2728] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:07:28.251878 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d50e27-4bbc-445f-a78d-7d675b0a1b01 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:28.257342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-61d50e27-4bbc-445f-a78d-7d675b0a1b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:28.285271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:28.336587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d50e27-4bbc-445f-a78d-7d675b0a1b01 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:28.340957 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e434f-e465-4672-85a0-a1f742430ff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:28.341658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e434f-e465-4672-85a0-a1f742430ff7 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:28.341940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e434f-e465-4672-85a0-a1f742430ff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:28.342214 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e434f-e465-4672-85a0-a1f742430ff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:28.345420 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-522e434f-e465-4672-85a0-a1f742430ff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:28.450754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e434f-e465-4672-85a0-a1f742430ff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:28.453288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e434f-e465-4672-85a0-a1f742430ff7 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:28.471169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1365/2729] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:07:35.906011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5d4c3-0288-4faa-ba5c-f8a75a4d89bc service cinder] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:35.924497 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5d4c3-0288-4faa-ba5c-f8a75a4d89bc service cinder] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:35.930365 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:35.931643 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:35.932034 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:35.932455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] PATH_INFO: `/v3/users/372fdaca2fef4a45bb675451a08ac49f` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:35.934328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] RBAC: Authorizing `identity:delete_user(user_id=372fdaca2fef4a45bb675451a08ac49f)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:35.936584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:36.007715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372fdaca2fef4a45bb675451a08ac49f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:36.021788 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372fdaca2fef4a45bb675451a08ac49f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:36.023208 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372fdaca2fef4a45bb675451a08ac49f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:36.049984 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372fdaca2fef4a45bb675451a08ac49f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:36.067477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372fdaca2fef4a45bb675451a08ac49f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:36.086566 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1365/2730] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:07:35 2022] DELETE /identity/v3/users/372fdaca2fef4a45bb675451a08ac49f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:07:36.088912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc232-2dc2-4014-8fca-0b6c1a1d4e63 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:36.089739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc232-2dc2-4014-8fca-0b6c1a1d4e63 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:36.089739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc232-2dc2-4014-8fca-0b6c1a1d4e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:36.111713 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-189dc232-2dc2-4014-8fca-0b6c1a1d4e63 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:36.113524 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-189dc232-2dc2-4014-8fca-0b6c1a1d4e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:36.246284 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1366/2731] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:07:36.263590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:36.265797 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:36.377646 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee1bf1-5a69-4a15-8363-db6f49e13599 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:36.385956 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66f2f2-d8de-4bd4-b013-7c223f4ac915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:36.388403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66f2f2-d8de-4bd4-b013-7c223f4ac915 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:36.388867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66f2f2-d8de-4bd4-b013-7c223f4ac915 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:36.389353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66f2f2-d8de-4bd4-b013-7c223f4ac915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:36.408180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66f2f2-d8de-4bd4-b013-7c223f4ac915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:36.411110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66f2f2-d8de-4bd4-b013-7c223f4ac915 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:36.430753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1366/2732] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:07:37.025490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dc232-2dc2-4014-8fca-0b6c1a1d4e63 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:37.028018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-189dc232-2dc2-4014-8fca-0b6c1a1d4e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:37.066035 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dc232-2dc2-4014-8fca-0b6c1a1d4e63 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:37.082330 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:37.084565 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:37.100547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:37.102230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] PATH_INFO: `/v3/projects/cd707eb241b64a05b637087f6a4870b4` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:37.106298 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd707eb241b64a05b637087f6a4870b4)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:37.111855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:37.219017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd707eb241b64a05b637087f6a4870b4', 'request_id': 'req-a959861a-ded0-4d78-b5b0-cf6a6c086874'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:37.231282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd707eb241b64a05b637087f6a4870b4', 'request_id': 'req-a959861a-ded0-4d78-b5b0-cf6a6c086874'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:37.259165 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd707eb241b64a05b637087f6a4870b4', 'request_id': 'req-a959861a-ded0-4d78-b5b0-cf6a6c086874'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:37.263743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:37.286948 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] The token cache is being invalidate because project cd707eb241b64a05b637087f6a4870b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:07:37.288081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:37.290245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-a959861a-ded0-4d78-b5b0-cf6a6c086874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:37.297299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1367/2733] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:07:37 2022] DELETE /identity/v3/projects/cd707eb241b64a05b637087f6a4870b4 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:07:37.304520 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2a9d1-e49d-4259-b756-c1ef6927e562 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:37.305432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2a9d1-e49d-4259-b756-c1ef6927e562 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:37.305432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2a9d1-e49d-4259-b756-c1ef6927e562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:37.329871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-9dc2a9d1-e49d-4259-b756-c1ef6927e562 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:37.331556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9dc2a9d1-e49d-4259-b756-c1ef6927e562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:37.388457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:37.454473 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1367/2734] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:07:37.470283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608e1e5-426d-4cbd-91f2-0201eedce137 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:37.470796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608e1e5-426d-4cbd-91f2-0201eedce137 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:37.471551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608e1e5-426d-4cbd-91f2-0201eedce137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:37.505848 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-4608e1e5-426d-4cbd-91f2-0201eedce137 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:37.508210 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-4608e1e5-426d-4cbd-91f2-0201eedce137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:37.528578 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1368/2735] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:07:37.532826 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2a9d1-e49d-4259-b756-c1ef6927e562 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:37.557268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2a9d1-e49d-4259-b756-c1ef6927e562 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:37.567176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958946b-3b17-430a-9c0f-291094f1b807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:37.568408 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958946b-3b17-430a-9c0f-291094f1b807 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:37.568929 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958946b-3b17-430a-9c0f-291094f1b807 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:37.569440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958946b-3b17-430a-9c0f-291094f1b807 admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:37.571202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958946b-3b17-430a-9c0f-291094f1b807 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:07:37.573473 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958946b-3b17-430a-9c0f-291094f1b807 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:37.582544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958946b-3b17-430a-9c0f-291094f1b807 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:37.613044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1368/2736] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:07:37.623283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608e1e5-426d-4cbd-91f2-0201eedce137 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:37.644399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608e1e5-426d-4cbd-91f2-0201eedce137 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:37.649128 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958946b-3b17-430a-9c0f-291094f1b807 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:37.651687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9958946b-3b17-430a-9c0f-291094f1b807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:37.653275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75187f11-d2c5-4464-8d72-6e1f057b176f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:37.654373 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75187f11-d2c5-4464-8d72-6e1f057b176f admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:37.655572 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75187f11-d2c5-4464-8d72-6e1f057b176f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:37.656491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75187f11-d2c5-4464-8d72-6e1f057b176f admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:37.658087 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75187f11-d2c5-4464-8d72-6e1f057b176f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:37.661765 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75187f11-d2c5-4464-8d72-6e1f057b176f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:37.677045 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958946b-3b17-430a-9c0f-291094f1b807 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:37.685049 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:37.686498 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:37.686878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:37.687474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] PATH_INFO: `/v3/users/5634da4012cc4d468f84409736c28e09` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:37.689622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5634da4012cc4d468f84409736c28e09)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:37.694100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:37.712300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1369/2737] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:07:37 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:07:37.722002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75187f11-d2c5-4464-8d72-6e1f057b176f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:37.784481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5634da4012cc4d468f84409736c28e09'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:37.802207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5634da4012cc4d468f84409736c28e09'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:37.803309 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5634da4012cc4d468f84409736c28e09'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:37.822427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5634da4012cc4d468f84409736c28e09'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:37.834162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75187f11-d2c5-4464-8d72-6e1f057b176f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:37.836987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5634da4012cc4d468f84409736c28e09'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:37.842051 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72dfdf-d0d7-4828-9b5b-42b65f28fa65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:37.842904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72dfdf-d0d7-4828-9b5b-42b65f28fa65 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:37.843282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72dfdf-d0d7-4828-9b5b-42b65f28fa65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:37.843656 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72dfdf-d0d7-4828-9b5b-42b65f28fa65 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:37.844855 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72dfdf-d0d7-4828-9b5b-42b65f28fa65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:37.847239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72dfdf-d0d7-4828-9b5b-42b65f28fa65 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:37.848873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1369/2738] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:07:37 2022] DELETE /identity/v3/users/5634da4012cc4d468f84409736c28e09 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:07:37.876070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1370/2739] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:07:37 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:07:37.883006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:37.885571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:38.013819 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c6730-29e6-4d2d-93bf-0235c550adf8 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:38.020548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07435e8-800b-4f10-8df6-6a3c8db03f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:38.021383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07435e8-800b-4f10-8df6-6a3c8db03f39 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:38.021678 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07435e8-800b-4f10-8df6-6a3c8db03f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:38.021991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07435e8-800b-4f10-8df6-6a3c8db03f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:38.023350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07435e8-800b-4f10-8df6-6a3c8db03f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:38.025700 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07435e8-800b-4f10-8df6-6a3c8db03f39 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:38.044686 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1370/2740] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:07:38.049748 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72dfdf-d0d7-4828-9b5b-42b65f28fa65 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:38.069466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72dfdf-d0d7-4828-9b5b-42b65f28fa65 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:38.081912 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ccdbd-6fc2-402f-aa4b-5672350243b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:38.083223 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ccdbd-6fc2-402f-aa4b-5672350243b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:38.083696 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ccdbd-6fc2-402f-aa4b-5672350243b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:38.084207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ccdbd-6fc2-402f-aa4b-5672350243b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:38.085655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ccdbd-6fc2-402f-aa4b-5672350243b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:38.088128 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ccdbd-6fc2-402f-aa4b-5672350243b0 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:38.102527 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1371/2741] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:07:38.108173 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07435e8-800b-4f10-8df6-6a3c8db03f39 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:38.123869 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07435e8-800b-4f10-8df6-6a3c8db03f39 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:38.131842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef93cb7-f054-497e-bed7-9c55c1da1963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:38.133166 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93cb7-f054-497e-bed7-9c55c1da1963 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:38.133539 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93cb7-f054-497e-bed7-9c55c1da1963 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:38.133882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93cb7-f054-497e-bed7-9c55c1da1963 admin admin] PATH_INFO: `/v3/projects/41ea12b35ccb471284d0bfde7d24089b/users/f649ce79152d4d23866dde7dbe067ea1/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:38.161619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ccdbd-6fc2-402f-aa4b-5672350243b0 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:38.165389 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-176ccdbd-6fc2-402f-aa4b-5672350243b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:38.166576 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef93cb7-f054-497e-bed7-9c55c1da1963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ea12b35ccb471284d0bfde7d24089b, user_id=f649ce79152d4d23866dde7dbe067ea1, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:38.169484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef93cb7-f054-497e-bed7-9c55c1da1963 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:38.192479 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ccdbd-6fc2-402f-aa4b-5672350243b0 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:38.198664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1371/2742] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:07:38 2022] PUT /identity/v3/projects/41ea12b35ccb471284d0bfde7d24089b/users/f649ce79152d4d23866dde7dbe067ea1/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:07:38.213232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2e8f4-b5c3-4582-bd88-b8ebc19a9f88 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:38.214093 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2e8f4-b5c3-4582-bd88-b8ebc19a9f88 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:38.214967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2e8f4-b5c3-4582-bd88-b8ebc19a9f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:38.238025 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e4e2e8f4-b5c3-4582-bd88-b8ebc19a9f88 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:38.239485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e4e2e8f4-b5c3-4582-bd88-b8ebc19a9f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:38.244746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:38.302397 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:38.304000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:38.304283 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:38.304603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] PATH_INFO: `/v3/projects/e08c8f78187d474a865ea712616685cc` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:38.305983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e08c8f78187d474a865ea712616685cc)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:38.307956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:38.321954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1372/2743] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:07:38.332168 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2e8f4-b5c3-4582-bd88-b8ebc19a9f88 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:38.333937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e4e2e8f4-b5c3-4582-bd88-b8ebc19a9f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:38.372454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:38.378190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e08c8f78187d474a865ea712616685cc', 'request_id': 'req-7ee085b8-04d2-45dd-b616-39c6b236ac97'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:38.393795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e08c8f78187d474a865ea712616685cc', 'request_id': 'req-7ee085b8-04d2-45dd-b616-39c6b236ac97'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:38.402594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e08c8f78187d474a865ea712616685cc', 'request_id': 'req-7ee085b8-04d2-45dd-b616-39c6b236ac97'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:38.421011 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] The token cache is being invalidate because project e08c8f78187d474a865ea712616685cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:07:38.421316 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:38.422155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:38.425151 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1372/2744] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:07:38 2022] DELETE /identity/v3/projects/e08c8f78187d474a865ea712616685cc => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:07:38.429897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:38.432058 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:38.443579 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2e8f4-b5c3-4582-bd88-b8ebc19a9f88 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:38.540075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a471b5-b0b5-4b9e-84fd-76b6edd203d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:38.541028 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a471b5-b0b5-4b9e-84fd-76b6edd203d0 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:38.541460 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a471b5-b0b5-4b9e-84fd-76b6edd203d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:38.541868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a471b5-b0b5-4b9e-84fd-76b6edd203d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:38.544379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee085b8-04d2-45dd-b616-39c6b236ac97 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:38.544723 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d7a471b5-b0b5-4b9e-84fd-76b6edd203d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:38.552204 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541105b-11e1-4bf6-b902-fb35aa80f139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:38.554115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541105b-11e1-4bf6-b902-fb35aa80f139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:38.554613 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541105b-11e1-4bf6-b902-fb35aa80f139 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:38.555122 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541105b-11e1-4bf6-b902-fb35aa80f139 admin admin] PATH_INFO: `/v3/users/5bad2a3e3d744290901be1c2227ba50f` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:38.557519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541105b-11e1-4bf6-b902-fb35aa80f139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bad2a3e3d744290901be1c2227ba50f)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:38.561387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541105b-11e1-4bf6-b902-fb35aa80f139 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:38.562449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a471b5-b0b5-4b9e-84fd-76b6edd203d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:38.564040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a471b5-b0b5-4b9e-84fd-76b6edd203d0 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:38.577227 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1373/2745] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:07:38.651467 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-1541105b-11e1-4bf6-b902-fb35aa80f139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bad2a3e3d744290901be1c2227ba50f'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:38.664596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-1541105b-11e1-4bf6-b902-fb35aa80f139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bad2a3e3d744290901be1c2227ba50f'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:38.678679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-1541105b-11e1-4bf6-b902-fb35aa80f139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bad2a3e3d744290901be1c2227ba50f'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:38.679312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-1541105b-11e1-4bf6-b902-fb35aa80f139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bad2a3e3d744290901be1c2227ba50f'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:38.705537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-1541105b-11e1-4bf6-b902-fb35aa80f139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bad2a3e3d744290901be1c2227ba50f'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:38.713841 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1373/2746] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:07:38 2022] DELETE /identity/v3/users/5bad2a3e3d744290901be1c2227ba50f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:07:38.952469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a471b5-b0b5-4b9e-84fd-76b6edd203d0 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:38.971715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a471b5-b0b5-4b9e-84fd-76b6edd203d0 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:39.075202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:39.077107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:39.077894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:39.078190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] PATH_INFO: `/v3/projects/1bd1c6a8ce4744f88fe546937f5c8c46` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:39.081502 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bd1c6a8ce4744f88fe546937f5c8c46)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:39.086837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:39.115713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:39.188836 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bd1c6a8ce4744f88fe546937f5c8c46', 'request_id': 'req-32759941-5223-4266-8179-25039e81f7b4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:39.219425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bd1c6a8ce4744f88fe546937f5c8c46', 'request_id': 'req-32759941-5223-4266-8179-25039e81f7b4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:39.238094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bd1c6a8ce4744f88fe546937f5c8c46', 'request_id': 'req-32759941-5223-4266-8179-25039e81f7b4'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:39.259353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] The token cache is being invalidate because project 1bd1c6a8ce4744f88fe546937f5c8c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:07:39.259977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:39.260886 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:07:39.262778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1374/2747] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:07:38 2022] DELETE /identity/v3/projects/1bd1c6a8ce4744f88fe546937f5c8c46 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:07:44.689611 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a7470-1be0-4f7a-a946-89e514382d43 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:44.690493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a7470-1be0-4f7a-a946-89e514382d43 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:44.690658 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a7470-1be0-4f7a-a946-89e514382d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:44.757275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e64a7470-1be0-4f7a-a946-89e514382d43 None None] MFA Rules not processed for user `f649ce79152d4d23866dde7dbe067ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:44.759553 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e64a7470-1be0-4f7a-a946-89e514382d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:44.849998 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:44.890387 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1374/2748] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:07:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:07:44.898496 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:44.900977 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:45.019668 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32759941-5223-4266-8179-25039e81f7b4 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:45.028115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffb8f1-c099-404b-867a-3eaefc7be8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:45.029634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb8f1-c099-404b-867a-3eaefc7be8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:45.029949 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb8f1-c099-404b-867a-3eaefc7be8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:45.030463 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb8f1-c099-404b-867a-3eaefc7be8a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:45.031867 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffb8f1-c099-404b-867a-3eaefc7be8a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:45.035775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffb8f1-c099-404b-867a-3eaefc7be8a2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:45.081289 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1375/2749] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:07:44 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:07:45.086744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a7470-1be0-4f7a-a946-89e514382d43 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:45.103816 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a7470-1be0-4f7a-a946-89e514382d43 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:45.193248 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9250f-742e-4f8b-8c10-f86c76ada8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:45.194019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9250f-742e-4f8b-8c10-f86c76ada8fa admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:45.194349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9250f-742e-4f8b-8c10-f86c76ada8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:45.194673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9250f-742e-4f8b-8c10-f86c76ada8fa admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:45.195824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9250f-742e-4f8b-8c10-f86c76ada8fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:45.197913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9250f-742e-4f8b-8c10-f86c76ada8fa admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:45.236381 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1375/2750] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:07:45 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:07:45.240782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffb8f1-c099-404b-867a-3eaefc7be8a2 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:45.253696 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffb8f1-c099-404b-867a-3eaefc7be8a2 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:45.258909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a83aa-064d-4e3c-ba21-ffa05e35665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:45.259757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a83aa-064d-4e3c-ba21-ffa05e35665d admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:45.259900 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a83aa-064d-4e3c-ba21-ffa05e35665d admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:45.260167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a83aa-064d-4e3c-ba21-ffa05e35665d admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:45.261210 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a83aa-064d-4e3c-ba21-ffa05e35665d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:45.263076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a83aa-064d-4e3c-ba21-ffa05e35665d admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:45.276957 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1376/2751] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:07:45.281800 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9250f-742e-4f8b-8c10-f86c76ada8fa admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:45.293413 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9250f-742e-4f8b-8c10-f86c76ada8fa admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:45.299612 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f8437-f54a-47fb-8652-eddc9466f42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:45.300758 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f8437-f54a-47fb-8652-eddc9466f42a admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:45.301240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f8437-f54a-47fb-8652-eddc9466f42a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:45.301597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f8437-f54a-47fb-8652-eddc9466f42a admin admin] PATH_INFO: `/v3/projects/e147291196c7411e9f9b566a7b777611/users/b3725298157749c29d105ec0a22930a0/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:45.321679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f8437-f54a-47fb-8652-eddc9466f42a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e147291196c7411e9f9b566a7b777611, user_id=b3725298157749c29d105ec0a22930a0, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:45.324589 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f8437-f54a-47fb-8652-eddc9466f42a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:45.340983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1376/2752] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:07:45 2022] PUT /identity/v3/projects/e147291196c7411e9f9b566a7b777611/users/b3725298157749c29d105ec0a22930a0/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:07:45.345753 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a83aa-064d-4e3c-ba21-ffa05e35665d admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:45.357257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a83aa-064d-4e3c-ba21-ffa05e35665d admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:45.437299 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9d893-d509-452c-90c8-e3de912d51da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:45.438375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9d893-d509-452c-90c8-e3de912d51da admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:45.438758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9d893-d509-452c-90c8-e3de912d51da admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:45.439138 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9d893-d509-452c-90c8-e3de912d51da admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:45.440513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9d893-d509-452c-90c8-e3de912d51da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:45.443304 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9d893-d509-452c-90c8-e3de912d51da admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:45.459480 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1377/2753] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:07:45.465269 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f8437-f54a-47fb-8652-eddc9466f42a admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:45.483126 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f8437-f54a-47fb-8652-eddc9466f42a admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:45.490375 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7541e3e-51ea-48b7-a0f5-cf09443fcb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:45.491445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7541e3e-51ea-48b7-a0f5-cf09443fcb2a admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:45.491826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7541e3e-51ea-48b7-a0f5-cf09443fcb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:45.492277 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7541e3e-51ea-48b7-a0f5-cf09443fcb2a admin admin] PATH_INFO: `/v3/projects/e147291196c7411e9f9b566a7b777611/users/b3725298157749c29d105ec0a22930a0/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:45.494697 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7541e3e-51ea-48b7-a0f5-cf09443fcb2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e147291196c7411e9f9b566a7b777611, user_id=b3725298157749c29d105ec0a22930a0, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:45.497719 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7541e3e-51ea-48b7-a0f5-cf09443fcb2a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:45.518271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1377/2754] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:07:45 2022] PUT /identity/v3/projects/e147291196c7411e9f9b566a7b777611/users/b3725298157749c29d105ec0a22930a0/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:07:51.641791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788d265-a1e4-41b8-9502-e1278464fa1d None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:51.643349 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788d265-a1e4-41b8-9502-e1278464fa1d None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:51.643812 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788d265-a1e4-41b8-9502-e1278464fa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:51.680144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:51.697146 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0788d265-a1e4-41b8-9502-e1278464fa1d None None] MFA Rules not processed for user `b3725298157749c29d105ec0a22930a0`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:51.698350 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0788d265-a1e4-41b8-9502-e1278464fa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:51.780059 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1378/2755] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:07:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:07:51.794238 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7541e3e-51ea-48b7-a0f5-cf09443fcb2a admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:51.796085 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f7541e3e-51ea-48b7-a0f5-cf09443fcb2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:51.896705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7541e3e-51ea-48b7-a0f5-cf09443fcb2a admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:51.900737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee7500-9a09-4616-9ab0-341138e38d69 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35e7720e3bf4c9e80bdefa6a1cb8296', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:51.901487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee7500-9a09-4616-9ab0-341138e38d69 service glance] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:51.901825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee7500-9a09-4616-9ab0-341138e38d69 service glance] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:51.902209 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee7500-9a09-4616-9ab0-341138e38d69 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:51.945990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:07:51.996941 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee7500-9a09-4616-9ab0-341138e38d69 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:51.998672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee7500-9a09-4616-9ab0-341138e38d69 service glance] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:52.025266 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1378/2756] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:07:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:07:52.258445 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305b24a-8918-4a98-9510-625351b716bc None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:52.258868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305b24a-8918-4a98-9510-625351b716bc None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:52.259241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305b24a-8918-4a98-9510-625351b716bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:52.354264 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e305b24a-8918-4a98-9510-625351b716bc None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:07:52.358622 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e305b24a-8918-4a98-9510-625351b716bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:52.485726 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1379/2757] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 21:07:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:07:52.555527 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee7500-9a09-4616-9ab0-341138e38d69 service glance] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:07:52.557303 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-08ee7500-9a09-4616-9ab0-341138e38d69 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:07:52.685197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee7500-9a09-4616-9ab0-341138e38d69 service glance] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:07:52.692706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675edae6-0f5f-473f-ad09-34d55e3c8c34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:07:52.693795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675edae6-0f5f-473f-ad09-34d55e3c8c34 service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:07:52.694302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675edae6-0f5f-473f-ad09-34d55e3c8c34 service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:07:52.694745 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675edae6-0f5f-473f-ad09-34d55e3c8c34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:07:52.710905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675edae6-0f5f-473f-ad09-34d55e3c8c34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:07:52.714074 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675edae6-0f5f-473f-ad09-34d55e3c8c34 service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:07:52.728027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1379/2758] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 21:07:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 21:08:02.479267 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=70 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:08:02.726515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=56 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:08:04.462590 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff79fc5-5644-4592-9f07-7db32e76ff67 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:08:04.463520 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff79fc5-5644-4592-9f07-7db32e76ff67 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:08:04.463520 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff79fc5-5644-4592-9f07-7db32e76ff67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:08:04.486414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-0ff79fc5-5644-4592-9f07-7db32e76ff67 None None] MFA Rules not processed for user `f649ce79152d4d23866dde7dbe067ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:08:04.487676 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0ff79fc5-5644-4592-9f07-7db32e76ff67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:08:04.498328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1380/2759] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:08:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:08:04.814487 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675edae6-0f5f-473f-ad09-34d55e3c8c34 service swift] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:08:04.818415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-675edae6-0f5f-473f-ad09-34d55e3c8c34 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:08:04.917543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675edae6-0f5f-473f-ad09-34d55e3c8c34 service swift] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:08:04.925027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461ab64-0613-4b84-a411-537c37c8bc74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:08:04.926160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ab64-0613-4b84-a411-537c37c8bc74 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:08:04.926648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ab64-0613-4b84-a411-537c37c8bc74 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:08:04.927146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461ab64-0613-4b84-a411-537c37c8bc74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:08:04.930344 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e461ab64-0613-4b84-a411-537c37c8bc74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:08:04.942083 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461ab64-0613-4b84-a411-537c37c8bc74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:08:04.943932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461ab64-0613-4b84-a411-537c37c8bc74 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:08:04.955262 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1380/2760] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:08:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:08:14.478416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:08:14.950150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=26 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:08:31.379597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff79fc5-5644-4592-9f07-7db32e76ff67 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:08:31.382247 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0ff79fc5-5644-4592-9f07-7db32e76ff67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:08:31.486939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff79fc5-5644-4592-9f07-7db32e76ff67 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:08:31.494317 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40ecd0-09d8-4d67-8f20-86818548820b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:08:31.495214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40ecd0-09d8-4d67-8f20-86818548820b service placement] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:08:31.495697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40ecd0-09d8-4d67-8f20-86818548820b service placement] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:08:31.496191 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40ecd0-09d8-4d67-8f20-86818548820b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:08:31.498609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-bb40ecd0-09d8-4d67-8f20-86818548820b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:08:31.514593 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40ecd0-09d8-4d67-8f20-86818548820b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:08:31.516279 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40ecd0-09d8-4d67-8f20-86818548820b service placement] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:08:31.528725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1381/2761] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:08:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:08:41.523535 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=32 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:09:08.716784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8865e-5aa0-4547-94fa-064647633ccc None None] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:08.717445 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8865e-5aa0-4547-94fa-064647633ccc None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:08.717908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8865e-5aa0-4547-94fa-064647633ccc None None] PATH_INFO: `/` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:08.720599 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1381/2762] 10.222.0.85 () {54 vars in 912 bytes} [Tue Jul 19 21:09:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 21:09:08.726935 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd2762-8d27-477c-810f-8f883ef2f399 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:08.727514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd2762-8d27-477c-810f-8f883ef2f399 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:08.727514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd2762-8d27-477c-810f-8f883ef2f399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:08.775511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-bdfd2762-8d27-477c-810f-8f883ef2f399 None None] MFA Rules not processed for user `fffebfc51dba47f68263c0bcd6b18d22`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:09:08.776709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-bdfd2762-8d27-477c-810f-8f883ef2f399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:08.787571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1382/2763] 10.222.0.85 () {58 vars in 1009 bytes} [Tue Jul 19 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:09:08.814517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8865e-5aa0-4547-94fa-064647633ccc None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:09:08.832240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8865e-5aa0-4547-94fa-064647633ccc None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:09:08.837536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40847e7f-0f2d-4127-bb42-fa03100492f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:09:08.838197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40847e7f-0f2d-4127-bb42-fa03100492f0 service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:08.838477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40847e7f-0f2d-4127-bb42-fa03100492f0 service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:08.838747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40847e7f-0f2d-4127-bb42-fa03100492f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:08.850405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40847e7f-0f2d-4127-bb42-fa03100492f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:09:08.852111 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40847e7f-0f2d-4127-bb42-fa03100492f0 service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:09:08.874706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1382/2764] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:09:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:09:09.147167 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681da0c-ab42-4eca-8482-291da73e2f53 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:09.147457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681da0c-ab42-4eca-8482-291da73e2f53 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:09.147722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681da0c-ab42-4eca-8482-291da73e2f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:09.168654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-9681da0c-ab42-4eca-8482-291da73e2f53 None None] MFA Rules not processed for user `e2f34a304e714f838d68dfdf214e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:09:09.171474 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9681da0c-ab42-4eca-8482-291da73e2f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:09.182974 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1383/2765] 10.222.0.85 () {58 vars in 967 bytes} [Tue Jul 19 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:09:09.241649 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40847e7f-0f2d-4127-bb42-fa03100492f0 service placement] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:09:09.254895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40847e7f-0f2d-4127-bb42-fa03100492f0 service placement] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:09:09.260287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d3bcb-4622-412b-9d29-d4bbd8dae11c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812342d822f54aaf98e324d5d4e56753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:09:09.261274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d3bcb-4622-412b-9d29-d4bbd8dae11c service swift] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:09.261618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d3bcb-4622-412b-9d29-d4bbd8dae11c service swift] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:09.261959 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d3bcb-4622-412b-9d29-d4bbd8dae11c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:09.272769 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d3bcb-4622-412b-9d29-d4bbd8dae11c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:09:09.274557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d3bcb-4622-412b-9d29-d4bbd8dae11c service swift] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:09:09.285030 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1383/2766] 10.222.0.85 () {60 vars in 1384 bytes} [Tue Jul 19 21:09:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 21:09:13.443827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681da0c-ab42-4eca-8482-291da73e2f53 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:09:13.445462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-9681da0c-ab42-4eca-8482-291da73e2f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:13.462244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681da0c-ab42-4eca-8482-291da73e2f53 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:09:13.480164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72e93f-79fb-4d47-99e4-1ffd458b108d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:09:13.481396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72e93f-79fb-4d47-99e4-1ffd458b108d service placement] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:13.481908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72e93f-79fb-4d47-99e4-1ffd458b108d service placement] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:13.482406 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72e93f-79fb-4d47-99e4-1ffd458b108d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:13.485940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-cb72e93f-79fb-4d47-99e4-1ffd458b108d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:13.508763 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72e93f-79fb-4d47-99e4-1ffd458b108d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:09:13.511148 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72e93f-79fb-4d47-99e4-1ffd458b108d service placement] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:09:13.527649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1384/2767] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:09:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:09:17.477325 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d3bcb-4622-412b-9d29-d4bbd8dae11c service swift] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:09:17.563059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d3bcb-4622-412b-9d29-d4bbd8dae11c service swift] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:09:17.572056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:09:17.573534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:17.574021 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:17.574490 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] PATH_INFO: `/v3/users/f649ce79152d4d23866dde7dbe067ea1` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:17.576655 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f649ce79152d4d23866dde7dbe067ea1)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:09:17.579596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:09:17.642870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f649ce79152d4d23866dde7dbe067ea1'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:17.652566 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f649ce79152d4d23866dde7dbe067ea1'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:17.660547 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f649ce79152d4d23866dde7dbe067ea1'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:17.661289 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f649ce79152d4d23866dde7dbe067ea1'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:17.674865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f649ce79152d4d23866dde7dbe067ea1'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:17.681477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1384/2768] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:09:17 2022] DELETE /identity/v3/users/f649ce79152d4d23866dde7dbe067ea1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:09:17.911061 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72e93f-79fb-4d47-99e4-1ffd458b108d service placement] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:09:17.922859 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72e93f-79fb-4d47-99e4-1ffd458b108d service placement] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:09:17.994703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:09:17.995607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:17.995894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:17.996179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] PATH_INFO: `/v3/projects/41ea12b35ccb471284d0bfde7d24089b` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:17.997570 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ea12b35ccb471284d0bfde7d24089b)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:09:17.999756 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:09:18.058232 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ea12b35ccb471284d0bfde7d24089b', 'request_id': 'req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.067501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ea12b35ccb471284d0bfde7d24089b', 'request_id': 'req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.080889 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ea12b35ccb471284d0bfde7d24089b', 'request_id': 'req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.096706 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] The token cache is being invalidate because project 41ea12b35ccb471284d0bfde7d24089b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:09:18.097305 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.098241 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.100549 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1385/2769] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:09:17 2022] DELETE /identity/v3/projects/41ea12b35ccb471284d0bfde7d24089b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:09:18.106645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:09:18.109805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:18.134130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c126c-0497-4ee9-8d15-e5e31478ff3a admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:09:18.206242 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:09:18.226340 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:09:18.226914 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:18.227186 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:18.227650 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] PATH_INFO: `/v3/users/b3725298157749c29d105ec0a22930a0` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:18.229231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3725298157749c29d105ec0a22930a0)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:09:18.231194 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:09:18.288061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3725298157749c29d105ec0a22930a0'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.297334 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3725298157749c29d105ec0a22930a0'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.305886 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3725298157749c29d105ec0a22930a0'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.306670 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3725298157749c29d105ec0a22930a0'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.320743 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3725298157749c29d105ec0a22930a0'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.328197 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1385/2770] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:09:18 2022] DELETE /identity/v3/users/b3725298157749c29d105ec0a22930a0 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:09:18.539102 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:09:18.541119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:18.556703 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604d68f-b06d-4de9-91e1-f3a11c8fca3c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:09:18.631233 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:09:18.644477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:09:18.645483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:18.645647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:18.646416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] PATH_INFO: `/v3/projects/e147291196c7411e9f9b566a7b777611` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:18.647368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e147291196c7411e9f9b566a7b777611)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:09:18.649639 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:09:18.704702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e147291196c7411e9f9b566a7b777611', 'request_id': 'req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.714133 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e147291196c7411e9f9b566a7b777611', 'request_id': 'req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.721490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e147291196c7411e9f9b566a7b777611', 'request_id': 'req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.732434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] The token cache is being invalidate because project e147291196c7411e9f9b566a7b777611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:09:18.732722 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.733333 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:09:18.734903 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1386/2771] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:09:18 2022] DELETE /identity/v3/projects/e147291196c7411e9f9b566a7b777611 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:09:28.323654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:09:28.729043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:09:48.782245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:09:48.783792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:48.844978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4392ed-5e33-442e-8ad0-9d9210a3d514 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:09:48.893300 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b11ebd-242a-4d41-9a33-515410618126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:09:48.893947 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b11ebd-242a-4d41-9a33-515410618126 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:48.894215 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b11ebd-242a-4d41-9a33-515410618126 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:48.894488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b11ebd-242a-4d41-9a33-515410618126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:48.896275 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f5b11ebd-242a-4d41-9a33-515410618126 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:48.969351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b11ebd-242a-4d41-9a33-515410618126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:09:48.970787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b11ebd-242a-4d41-9a33-515410618126 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:09:48.981925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1386/2772] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:09:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:09:51.599932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:09:51.601620 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:51.664890 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7e19b-e2d7-4107-8efd-1dc7aafbe254 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:09:51.668625 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef767d80-6855-4246-85e5-c37c3b2fd430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:09:51.669281 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef767d80-6855-4246-85e5-c37c3b2fd430 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:09:51.670174 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef767d80-6855-4246-85e5-c37c3b2fd430 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:09:51.670454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef767d80-6855-4246-85e5-c37c3b2fd430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:09:51.672315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ef767d80-6855-4246-85e5-c37c3b2fd430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:09:51.686635 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef767d80-6855-4246-85e5-c37c3b2fd430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:09:51.688080 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef767d80-6855-4246-85e5-c37c3b2fd430 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:09:51.700299 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1387/2773] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:09:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:09:58.976962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=74 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:10:01.694677 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=26 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:10:32.211139 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b11ebd-242a-4d41-9a33-515410618126 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:10:32.213098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-f5b11ebd-242a-4d41-9a33-515410618126 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:10:32.230263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b11ebd-242a-4d41-9a33-515410618126 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:10:32.234412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704184fa-a210-4bfd-a3a9-f20be763cf6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:10:32.235522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704184fa-a210-4bfd-a3a9-f20be763cf6c service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:10:32.235714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704184fa-a210-4bfd-a3a9-f20be763cf6c service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:10:32.236022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704184fa-a210-4bfd-a3a9-f20be763cf6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:10:32.238026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-704184fa-a210-4bfd-a3a9-f20be763cf6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:10:32.250125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704184fa-a210-4bfd-a3a9-f20be763cf6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:10:32.251614 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704184fa-a210-4bfd-a3a9-f20be763cf6c service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:10:32.260672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1387/2774] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:10:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:10:42.256553 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=6 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:11:14.551669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2c9c-e110-414e-b7a7-9e1146cd87eb None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:11:14.552229 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2c9c-e110-414e-b7a7-9e1146cd87eb None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:11:14.552371 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2c9c-e110-414e-b7a7-9e1146cd87eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:11:14.570533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-a5ad2c9c-e110-414e-b7a7-9e1146cd87eb None None] MFA Rules not processed for user `187add5855314ff5b802b74297a6e7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:11:14.571732 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a5ad2c9c-e110-414e-b7a7-9e1146cd87eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:11:14.628086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1388/2775] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:11:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:11:14.945438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704184fa-a210-4bfd-a3a9-f20be763cf6c service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:11:14.955243 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704184fa-a210-4bfd-a3a9-f20be763cf6c service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:11:14.958874 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9305c0-bc77-42a2-a4ac-ba1cfae2448b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:11:14.959509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9305c0-bc77-42a2-a4ac-ba1cfae2448b service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:11:14.959790 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9305c0-bc77-42a2-a4ac-ba1cfae2448b service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:11:14.960065 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9305c0-bc77-42a2-a4ac-ba1cfae2448b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:11:14.961890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-2e9305c0-bc77-42a2-a4ac-ba1cfae2448b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:11:14.971577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9305c0-bc77-42a2-a4ac-ba1cfae2448b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:11:14.973064 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9305c0-bc77-42a2-a4ac-ba1cfae2448b service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:11:14.981105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1388/2776] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:11:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:11:21.989225 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad2c9c-e110-414e-b7a7-9e1146cd87eb None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:11:21.991508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-a5ad2c9c-e110-414e-b7a7-9e1146cd87eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:11:22.012180 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad2c9c-e110-414e-b7a7-9e1146cd87eb None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:11:22.018612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce88b85-3a81-453b-b641-bfe2d6e5b5fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:11:22.019307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce88b85-3a81-453b-b641-bfe2d6e5b5fb service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:11:22.019628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce88b85-3a81-453b-b641-bfe2d6e5b5fb service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:11:22.019942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce88b85-3a81-453b-b641-bfe2d6e5b5fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:11:22.021840 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-fce88b85-3a81-453b-b641-bfe2d6e5b5fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:11:22.034079 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce88b85-3a81-453b-b641-bfe2d6e5b5fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:11:22.038067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce88b85-3a81-453b-b641-bfe2d6e5b5fb service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:11:22.048198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1389/2777] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:11:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:11:24.977462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=6 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:11:32.043501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=31 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:13.084416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9305c0-bc77-42a2-a4ac-ba1cfae2448b service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:13.181013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9305c0-bc77-42a2-a4ac-ba1cfae2448b service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:13.187657 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef855f0-7c79-4e81-bc7d-4e2fb3a02ffa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:13.188353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef855f0-7c79-4e81-bc7d-4e2fb3a02ffa service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:13.188634 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef855f0-7c79-4e81-bc7d-4e2fb3a02ffa service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:13.188907 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef855f0-7c79-4e81-bc7d-4e2fb3a02ffa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:13.190931 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-aef855f0-7c79-4e81-bc7d-4e2fb3a02ffa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:13.231979 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef855f0-7c79-4e81-bc7d-4e2fb3a02ffa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:13.233692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef855f0-7c79-4e81-bc7d-4e2fb3a02ffa service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:13.248438 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1389/2778] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:13:18.895434 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce88b85-3a81-453b-b641-bfe2d6e5b5fb service nova] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:18.925713 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce88b85-3a81-453b-b641-bfe2d6e5b5fb service nova] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:18.929385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9489e-77be-4c8e-a315-44c40aede261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:18.930075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9489e-77be-4c8e-a315-44c40aede261 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:18.930358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9489e-77be-4c8e-a315-44c40aede261 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:18.930634 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9489e-77be-4c8e-a315-44c40aede261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:18.932501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-89b9489e-77be-4c8e-a315-44c40aede261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:19.021443 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9489e-77be-4c8e-a315-44c40aede261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:19.022878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9489e-77be-4c8e-a315-44c40aede261 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:19.042664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1390/2779] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:13:22.112150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef855f0-7c79-4e81-bc7d-4e2fb3a02ffa service nova] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:22.113701 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-aef855f0-7c79-4e81-bc7d-4e2fb3a02ffa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:22.179508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9489e-77be-4c8e-a315-44c40aede261 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:22.181052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-89b9489e-77be-4c8e-a315-44c40aede261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:22.197803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9489e-77be-4c8e-a315-44c40aede261 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:22.207725 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16be88-6a45-4bc3-b6a3-a8a7d9d1c38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:22.208571 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16be88-6a45-4bc3-b6a3-a8a7d9d1c38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:22.208861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16be88-6a45-4bc3-b6a3-a8a7d9d1c38d admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:22.209135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16be88-6a45-4bc3-b6a3-a8a7d9d1c38d admin admin] PATH_INFO: `/v3/users/187add5855314ff5b802b74297a6e7b0` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:22.210364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16be88-6a45-4bc3-b6a3-a8a7d9d1c38d admin admin] RBAC: Authorizing `identity:delete_user(user_id=187add5855314ff5b802b74297a6e7b0)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:22.213290 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16be88-6a45-4bc3-b6a3-a8a7d9d1c38d admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:22.226448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef855f0-7c79-4e81-bc7d-4e2fb3a02ffa service nova] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:22.236607 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cb9a5-443b-409d-8e32-30276a9dd2ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:22.238384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cb9a5-443b-409d-8e32-30276a9dd2ac service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:22.238725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cb9a5-443b-409d-8e32-30276a9dd2ac service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:22.238995 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cb9a5-443b-409d-8e32-30276a9dd2ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:22.241019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7e5cb9a5-443b-409d-8e32-30276a9dd2ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:22.255728 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5cb9a5-443b-409d-8e32-30276a9dd2ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:22.257539 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5cb9a5-443b-409d-8e32-30276a9dd2ac service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:22.271326 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1390/2780] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:13:22.275442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fd16be88-6a45-4bc3-b6a3-a8a7d9d1c38d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '187add5855314ff5b802b74297a6e7b0'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:22.285501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fd16be88-6a45-4bc3-b6a3-a8a7d9d1c38d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '187add5855314ff5b802b74297a6e7b0'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:22.289176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fd16be88-6a45-4bc3-b6a3-a8a7d9d1c38d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '187add5855314ff5b802b74297a6e7b0'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:22.309467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fd16be88-6a45-4bc3-b6a3-a8a7d9d1c38d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '187add5855314ff5b802b74297a6e7b0'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:22.318588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fd16be88-6a45-4bc3-b6a3-a8a7d9d1c38d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '187add5855314ff5b802b74297a6e7b0'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:22.327220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1391/2781] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:13:22 2022] DELETE /identity/v3/users/187add5855314ff5b802b74297a6e7b0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:22.508428 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cb9a5-443b-409d-8e32-30276a9dd2ac service placement] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:22.519674 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cb9a5-443b-409d-8e32-30276a9dd2ac service placement] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:22.563858 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:22.579896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:22.580802 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:22.581081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:22.581353 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] PATH_INFO: `/v3/projects/cb315d23f40a4d1bbe9828c1d3a4e5a7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:22.582630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb315d23f40a4d1bbe9828c1d3a4e5a7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:22.584551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:22.631184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb315d23f40a4d1bbe9828c1d3a4e5a7', 'request_id': 'req-77c332f2-58de-49ba-be01-b41b7106f48c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:22.639213 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb315d23f40a4d1bbe9828c1d3a4e5a7', 'request_id': 'req-77c332f2-58de-49ba-be01-b41b7106f48c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:22.645371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb315d23f40a4d1bbe9828c1d3a4e5a7', 'request_id': 'req-77c332f2-58de-49ba-be01-b41b7106f48c'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:22.655653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] The token cache is being invalidate because project cb315d23f40a4d1bbe9828c1d3a4e5a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:13:22.655935 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:22.656606 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-77c332f2-58de-49ba-be01-b41b7106f48c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:22.658156 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1391/2782] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:13:22 2022] DELETE /identity/v3/projects/cb315d23f40a4d1bbe9828c1d3a4e5a7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:22.663143 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340605fe-5cd3-44b0-9bc9-0afaa884c6fd None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:22.663537 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340605fe-5cd3-44b0-9bc9-0afaa884c6fd None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:22.663883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340605fe-5cd3-44b0-9bc9-0afaa884c6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:22.685681 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:22.722966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-340605fe-5cd3-44b0-9bc9-0afaa884c6fd None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:22.724199 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-340605fe-5cd3-44b0-9bc9-0afaa884c6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:22.783020 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1392/2783] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:22.791734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d322db-7940-47a9-be73-14cd6a39a6c4 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:22.792047 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d322db-7940-47a9-be73-14cd6a39a6c4 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:22.792343 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d322db-7940-47a9-be73-14cd6a39a6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:22.811422 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e8d322db-7940-47a9-be73-14cd6a39a6c4 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:22.812706 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e8d322db-7940-47a9-be73-14cd6a39a6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:22.820908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1392/2784] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:22.825376 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340605fe-5cd3-44b0-9bc9-0afaa884c6fd None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:22.835002 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340605fe-5cd3-44b0-9bc9-0afaa884c6fd None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:22.839765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6c75e-c8a7-48f5-81d2-d2f3b88bc6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:22.840491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6c75e-c8a7-48f5-81d2-d2f3b88bc6fd admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:22.840782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6c75e-c8a7-48f5-81d2-d2f3b88bc6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:22.841065 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6c75e-c8a7-48f5-81d2-d2f3b88bc6fd admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:22.841792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6c75e-c8a7-48f5-81d2-d2f3b88bc6fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:13:22.842100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6c75e-c8a7-48f5-81d2-d2f3b88bc6fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:22.843699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6c75e-c8a7-48f5-81d2-d2f3b88bc6fd admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:22.855770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1393/2785] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:13:22.860204 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d322db-7940-47a9-be73-14cd6a39a6c4 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:22.870544 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d322db-7940-47a9-be73-14cd6a39a6c4 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:22.875668 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1cc5d-d962-4b09-8427-5f0c07f5a767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:22.876350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1cc5d-d962-4b09-8427-5f0c07f5a767 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:22.876638 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1cc5d-d962-4b09-8427-5f0c07f5a767 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:22.876916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1cc5d-d962-4b09-8427-5f0c07f5a767 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:22.877618 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1cc5d-d962-4b09-8427-5f0c07f5a767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:22.879576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1cc5d-d962-4b09-8427-5f0c07f5a767 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:22.908777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1393/2786] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:13:22 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:13:22.910475 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6c75e-c8a7-48f5-81d2-d2f3b88bc6fd admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:22.921319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6c75e-c8a7-48f5-81d2-d2f3b88bc6fd admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:22.978818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a84b96-816a-4f29-8a4e-6dc3b302b144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:22.979485 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a84b96-816a-4f29-8a4e-6dc3b302b144 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:22.979798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a84b96-816a-4f29-8a4e-6dc3b302b144 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:22.980076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a84b96-816a-4f29-8a4e-6dc3b302b144 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:22.980793 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a84b96-816a-4f29-8a4e-6dc3b302b144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:22.982704 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a84b96-816a-4f29-8a4e-6dc3b302b144 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:23.004018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1394/2787] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:13:22 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:13:23.008163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1cc5d-d962-4b09-8427-5f0c07f5a767 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:23.018879 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1cc5d-d962-4b09-8427-5f0c07f5a767 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:23.023577 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0e7e6-54d2-476e-9d81-64153c0a1ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:23.024230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0e7e6-54d2-476e-9d81-64153c0a1ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.024543 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0e7e6-54d2-476e-9d81-64153c0a1ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.024851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0e7e6-54d2-476e-9d81-64153c0a1ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.025626 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0e7e6-54d2-476e-9d81-64153c0a1ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:23.027434 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0e7e6-54d2-476e-9d81-64153c0a1ba9 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:23.037916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1394/2788] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:13:23.042642 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a84b96-816a-4f29-8a4e-6dc3b302b144 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:23.052885 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a84b96-816a-4f29-8a4e-6dc3b302b144 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:23.058255 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eca7b2-b304-4036-9958-b97492b1485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:23.058914 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eca7b2-b304-4036-9958-b97492b1485f admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.059197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eca7b2-b304-4036-9958-b97492b1485f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.059476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eca7b2-b304-4036-9958-b97492b1485f admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.060206 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eca7b2-b304-4036-9958-b97492b1485f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:23.061820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eca7b2-b304-4036-9958-b97492b1485f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:23.072177 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1395/2789] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:13:23.076742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0e7e6-54d2-476e-9d81-64153c0a1ba9 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:23.087281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0e7e6-54d2-476e-9d81-64153c0a1ba9 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:23.091942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef67f50-13a8-457e-8030-690b90d917e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:23.092847 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef67f50-13a8-457e-8030-690b90d917e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.093136 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef67f50-13a8-457e-8030-690b90d917e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.093442 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef67f50-13a8-457e-8030-690b90d917e0 admin admin] PATH_INFO: `/v3/projects/ec70022a88b74705af3dcefe89b5082f/users/7d8f2dea1c734c9b9a7f36088a624bfb/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.108897 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef67f50-13a8-457e-8030-690b90d917e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec70022a88b74705af3dcefe89b5082f, user_id=7d8f2dea1c734c9b9a7f36088a624bfb, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:23.111533 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef67f50-13a8-457e-8030-690b90d917e0 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:23.125116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1395/2790] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:13:23 2022] PUT /identity/v3/projects/ec70022a88b74705af3dcefe89b5082f/users/7d8f2dea1c734c9b9a7f36088a624bfb/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:13:23.136278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b58da-46f6-4654-9270-cd88a103fa9a None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.136610 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b58da-46f6-4654-9270-cd88a103fa9a None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.136893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b58da-46f6-4654-9270-cd88a103fa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.163574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-589b58da-46f6-4654-9270-cd88a103fa9a None None] MFA Rules not processed for user `7d8f2dea1c734c9b9a7f36088a624bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:23.165441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-589b58da-46f6-4654-9270-cd88a103fa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:23.211432 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:23.236532 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1396/2791] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:23 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:13:23.241154 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef67f50-13a8-457e-8030-690b90d917e0 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:23.306496 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef67f50-13a8-457e-8030-690b90d917e0 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:23.311217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213b54e-4b65-496e-8653-624692738a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:23.311865 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213b54e-4b65-496e-8653-624692738a90 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.312145 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213b54e-4b65-496e-8653-624692738a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.312458 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213b54e-4b65-496e-8653-624692738a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.313144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213b54e-4b65-496e-8653-624692738a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:23.315025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213b54e-4b65-496e-8653-624692738a90 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:23.329992 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:23.344301 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1396/2792] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:13:23 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:13:23.348515 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b58da-46f6-4654-9270-cd88a103fa9a None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:23.360100 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b58da-46f6-4654-9270-cd88a103fa9a None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:23.416690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd358b5-8adb-448a-bca6-3a80f49268f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:23.417362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd358b5-8adb-448a-bca6-3a80f49268f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.417650 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd358b5-8adb-448a-bca6-3a80f49268f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.417930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd358b5-8adb-448a-bca6-3a80f49268f1 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.418621 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd358b5-8adb-448a-bca6-3a80f49268f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:23.420569 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd358b5-8adb-448a-bca6-3a80f49268f1 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:23.441566 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1397/2793] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:13:23 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:13:23.446061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213b54e-4b65-496e-8653-624692738a90 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:23.456930 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213b54e-4b65-496e-8653-624692738a90 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:23.462274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909bfe0-19b7-42c0-b1ba-d73c0fb0ea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:23.462928 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909bfe0-19b7-42c0-b1ba-d73c0fb0ea97 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.463219 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909bfe0-19b7-42c0-b1ba-d73c0fb0ea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.463500 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909bfe0-19b7-42c0-b1ba-d73c0fb0ea97 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.464229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909bfe0-19b7-42c0-b1ba-d73c0fb0ea97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:23.465870 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909bfe0-19b7-42c0-b1ba-d73c0fb0ea97 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:23.477399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1397/2794] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:13:23.481640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd358b5-8adb-448a-bca6-3a80f49268f1 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:23.491720 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd358b5-8adb-448a-bca6-3a80f49268f1 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:23.496740 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce168c-3524-44fa-9ce9-fe57afda37da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:23.497638 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce168c-3524-44fa-9ce9-fe57afda37da admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.497913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce168c-3524-44fa-9ce9-fe57afda37da admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.498188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce168c-3524-44fa-9ce9-fe57afda37da admin admin] PATH_INFO: `/v3/projects/d36847c50dd54e449d605c6787b60576/users/b6ae59d449a14b7ea3996915bbdede32/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.514420 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce168c-3524-44fa-9ce9-fe57afda37da admin admin] RBAC: Authorizing `identity:create_grant(project_id=d36847c50dd54e449d605c6787b60576, user_id=b6ae59d449a14b7ea3996915bbdede32, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:23.517107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce168c-3524-44fa-9ce9-fe57afda37da admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:23.531966 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1398/2795] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:13:23 2022] PUT /identity/v3/projects/d36847c50dd54e449d605c6787b60576/users/b6ae59d449a14b7ea3996915bbdede32/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:23.535990 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909bfe0-19b7-42c0-b1ba-d73c0fb0ea97 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:23.546062 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909bfe0-19b7-42c0-b1ba-d73c0fb0ea97 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:23.602945 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04476b0-e7ea-41f0-b53a-6a5e5d5619af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:23.603593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04476b0-e7ea-41f0-b53a-6a5e5d5619af admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.603883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04476b0-e7ea-41f0-b53a-6a5e5d5619af admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.604160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04476b0-e7ea-41f0-b53a-6a5e5d5619af admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.604915 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04476b0-e7ea-41f0-b53a-6a5e5d5619af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:23.606600 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04476b0-e7ea-41f0-b53a-6a5e5d5619af admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:23.617367 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1398/2796] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:13:23.622320 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce168c-3524-44fa-9ce9-fe57afda37da admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:23.633414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce168c-3524-44fa-9ce9-fe57afda37da admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:23.638266 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57c5fc-d6b8-4ba6-b6fd-2578194afb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:23.639147 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57c5fc-d6b8-4ba6-b6fd-2578194afb29 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.639435 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57c5fc-d6b8-4ba6-b6fd-2578194afb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.639716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57c5fc-d6b8-4ba6-b6fd-2578194afb29 admin admin] PATH_INFO: `/v3/projects/d36847c50dd54e449d605c6787b60576/users/b6ae59d449a14b7ea3996915bbdede32/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.641842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57c5fc-d6b8-4ba6-b6fd-2578194afb29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d36847c50dd54e449d605c6787b60576, user_id=b6ae59d449a14b7ea3996915bbdede32, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:23.644517 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57c5fc-d6b8-4ba6-b6fd-2578194afb29 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:23.655553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1399/2797] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:13:23 2022] PUT /identity/v3/projects/d36847c50dd54e449d605c6787b60576/users/b6ae59d449a14b7ea3996915bbdede32/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:23.665794 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aacb4e-01bc-49f0-9283-7b1d3bbce197 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.666080 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aacb4e-01bc-49f0-9283-7b1d3bbce197 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.666358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aacb4e-01bc-49f0-9283-7b1d3bbce197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.694453 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-19aacb4e-01bc-49f0-9283-7b1d3bbce197 None None] MFA Rules not processed for user `b6ae59d449a14b7ea3996915bbdede32`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:23.696382 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-19aacb4e-01bc-49f0-9283-7b1d3bbce197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:23.765388 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1399/2798] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:23 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:23.777981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57c5fc-d6b8-4ba6-b6fd-2578194afb29 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:23.779728 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0e57c5fc-d6b8-4ba6-b6fd-2578194afb29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:23.859536 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57c5fc-d6b8-4ba6-b6fd-2578194afb29 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:23.863301 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0fbbc-accd-4972-95a5-361f4979edfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:23.863968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0fbbc-accd-4972-95a5-361f4979edfc service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:23.864258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0fbbc-accd-4972-95a5-361f4979edfc service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:23.864574 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0fbbc-accd-4972-95a5-361f4979edfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:23.875216 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0fbbc-accd-4972-95a5-361f4979edfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:23.876770 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0fbbc-accd-4972-95a5-361f4979edfc service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:23.887784 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1400/2799] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:13:24.290350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aacb4e-01bc-49f0-9283-7b1d3bbce197 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:24.354219 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aacb4e-01bc-49f0-9283-7b1d3bbce197 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:24.358783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:24.359648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:24.359929 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:24.360206 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] PATH_INFO: `/v3/users/7d8f2dea1c734c9b9a7f36088a624bfb` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:24.361435 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d8f2dea1c734c9b9a7f36088a624bfb)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:24.363342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:24.369169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=100 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:24.411218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d8f2dea1c734c9b9a7f36088a624bfb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:24.419281 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d8f2dea1c734c9b9a7f36088a624bfb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:24.426786 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d8f2dea1c734c9b9a7f36088a624bfb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:24.427435 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d8f2dea1c734c9b9a7f36088a624bfb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:24.440714 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d8f2dea1c734c9b9a7f36088a624bfb'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:24.447527 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1400/2800] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:13:24 2022] DELETE /identity/v3/users/7d8f2dea1c734c9b9a7f36088a624bfb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:13:24.452792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eadc40-bdaa-4a91-9cfa-68ac55d2eca8 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:24.453293 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eadc40-bdaa-4a91-9cfa-68ac55d2eca8 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:24.453410 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eadc40-bdaa-4a91-9cfa-68ac55d2eca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:24.470585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-78eadc40-bdaa-4a91-9cfa-68ac55d2eca8 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:24.471791 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-78eadc40-bdaa-4a91-9cfa-68ac55d2eca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:24.483505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:24.532511 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1401/2801] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:24.542938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:24.544513 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:24.618551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ffa13-08c9-4f30-acd8-f453858caa3d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:24.622404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2c7ab-6da4-4c93-b86a-33348ea6ee70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:24.623013 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2c7ab-6da4-4c93-b86a-33348ea6ee70 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:24.623296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2c7ab-6da4-4c93-b86a-33348ea6ee70 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:24.623570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2c7ab-6da4-4c93-b86a-33348ea6ee70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:24.633045 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2c7ab-6da4-4c93-b86a-33348ea6ee70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:24.634477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2c7ab-6da4-4c93-b86a-33348ea6ee70 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:24.644223 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1401/2802] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:13:24.932107 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eadc40-bdaa-4a91-9cfa-68ac55d2eca8 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:24.933752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-78eadc40-bdaa-4a91-9cfa-68ac55d2eca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:24.949175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eadc40-bdaa-4a91-9cfa-68ac55d2eca8 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:24.954197 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:24.955078 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:24.955375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:24.955648 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] PATH_INFO: `/v3/projects/ec70022a88b74705af3dcefe89b5082f` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:24.956874 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec70022a88b74705af3dcefe89b5082f)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:24.958782 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:25.006684 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec70022a88b74705af3dcefe89b5082f', 'request_id': 'req-5089926a-0c5e-4c57-a841-89d43f9aff4f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.016652 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec70022a88b74705af3dcefe89b5082f', 'request_id': 'req-5089926a-0c5e-4c57-a841-89d43f9aff4f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.025214 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec70022a88b74705af3dcefe89b5082f', 'request_id': 'req-5089926a-0c5e-4c57-a841-89d43f9aff4f'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.036006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] The token cache is being invalidate because project ec70022a88b74705af3dcefe89b5082f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:13:25.036292 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.036969 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.038548 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1402/2803] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:13:24 2022] DELETE /identity/v3/projects/ec70022a88b74705af3dcefe89b5082f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:25.042518 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2c7ab-6da4-4c93-b86a-33348ea6ee70 service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:25.052580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2c7ab-6da4-4c93-b86a-33348ea6ee70 service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:25.108285 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446d66b-3f1b-4138-9f1b-e1d8bd79193b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:25.109181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446d66b-3f1b-4138-9f1b-e1d8bd79193b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:25.109462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446d66b-3f1b-4138-9f1b-e1d8bd79193b admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:25.109737 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446d66b-3f1b-4138-9f1b-e1d8bd79193b admin admin] PATH_INFO: `/v3/users/b6ae59d449a14b7ea3996915bbdede32` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:25.110942 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446d66b-3f1b-4138-9f1b-e1d8bd79193b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ae59d449a14b7ea3996915bbdede32)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:25.112918 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446d66b-3f1b-4138-9f1b-e1d8bd79193b admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:25.135371 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:25.162027 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9446d66b-3f1b-4138-9f1b-e1d8bd79193b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ae59d449a14b7ea3996915bbdede32'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.170472 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9446d66b-3f1b-4138-9f1b-e1d8bd79193b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ae59d449a14b7ea3996915bbdede32'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.177549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9446d66b-3f1b-4138-9f1b-e1d8bd79193b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ae59d449a14b7ea3996915bbdede32'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.178130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9446d66b-3f1b-4138-9f1b-e1d8bd79193b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ae59d449a14b7ea3996915bbdede32'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.190379 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-9446d66b-3f1b-4138-9f1b-e1d8bd79193b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ae59d449a14b7ea3996915bbdede32'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.197451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1402/2804] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:13:25 2022] DELETE /identity/v3/users/b6ae59d449a14b7ea3996915bbdede32 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:25.500089 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:25.501605 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:25.516080 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089926a-0c5e-4c57-a841-89d43f9aff4f admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:25.575271 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:25.576261 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:25.576599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:25.576882 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] PATH_INFO: `/v3/projects/d36847c50dd54e449d605c6787b60576` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:25.578088 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d36847c50dd54e449d605c6787b60576)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:25.580038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:25.624318 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:25.628701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd36847c50dd54e449d605c6787b60576', 'request_id': 'req-6ce3705d-0c5d-4218-9dff-1861ff8a634d'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.637205 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd36847c50dd54e449d605c6787b60576', 'request_id': 'req-6ce3705d-0c5d-4218-9dff-1861ff8a634d'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.643778 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd36847c50dd54e449d605c6787b60576', 'request_id': 'req-6ce3705d-0c5d-4218-9dff-1861ff8a634d'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.655022 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] The token cache is being invalidate because project d36847c50dd54e449d605c6787b60576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:13:25.655533 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.656266 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-6ce3705d-0c5d-4218-9dff-1861ff8a634d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:25.657918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1403/2805] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:13:25 2022] DELETE /identity/v3/projects/d36847c50dd54e449d605c6787b60576 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:25.662895 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a309065-0eda-4e9b-b23a-fec8f2b8001a None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:25.663383 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a309065-0eda-4e9b-b23a-fec8f2b8001a None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:25.663505 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a309065-0eda-4e9b-b23a-fec8f2b8001a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:25.682276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-6a309065-0eda-4e9b-b23a-fec8f2b8001a None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:25.683514 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6a309065-0eda-4e9b-b23a-fec8f2b8001a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:25.749705 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1403/2806] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:25.758437 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805d5d8-2b36-4aca-a12d-d2c3b4cc7687 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:25.758724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805d5d8-2b36-4aca-a12d-d2c3b4cc7687 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:25.759006 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805d5d8-2b36-4aca-a12d-d2c3b4cc7687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:25.778246 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-e805d5d8-2b36-4aca-a12d-d2c3b4cc7687 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:25.779451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-e805d5d8-2b36-4aca-a12d-d2c3b4cc7687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:25.787626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1404/2807] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:25.791936 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a309065-0eda-4e9b-b23a-fec8f2b8001a None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:25.803561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a309065-0eda-4e9b-b23a-fec8f2b8001a None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:25.808710 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e9ab6-4bec-413d-8bf7-bdae036b1da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:25.809358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e9ab6-4bec-413d-8bf7-bdae036b1da3 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:25.809638 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e9ab6-4bec-413d-8bf7-bdae036b1da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:25.809960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e9ab6-4bec-413d-8bf7-bdae036b1da3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:25.810662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e9ab6-4bec-413d-8bf7-bdae036b1da3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:13:25.810970 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e9ab6-4bec-413d-8bf7-bdae036b1da3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:25.812580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e9ab6-4bec-413d-8bf7-bdae036b1da3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:25.825055 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1404/2808] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:13:25.829667 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805d5d8-2b36-4aca-a12d-d2c3b4cc7687 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:25.839125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805d5d8-2b36-4aca-a12d-d2c3b4cc7687 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:25.843878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4552ab-990a-4afa-9a08-040e44ad44c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:25.844567 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4552ab-990a-4afa-9a08-040e44ad44c6 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:25.844853 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4552ab-990a-4afa-9a08-040e44ad44c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:25.845131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4552ab-990a-4afa-9a08-040e44ad44c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:25.845824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4552ab-990a-4afa-9a08-040e44ad44c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:25.847735 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4552ab-990a-4afa-9a08-040e44ad44c6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:25.874584 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1405/2809] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:13:25 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:13:25.878995 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e9ab6-4bec-413d-8bf7-bdae036b1da3 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:25.891436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e9ab6-4bec-413d-8bf7-bdae036b1da3 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:25.949792 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769b577-c0b5-4f55-aa34-99328b749f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:25.950459 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769b577-c0b5-4f55-aa34-99328b749f3e admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:25.950742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769b577-c0b5-4f55-aa34-99328b749f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:25.951022 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769b577-c0b5-4f55-aa34-99328b749f3e admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:25.951711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769b577-c0b5-4f55-aa34-99328b749f3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:25.953662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769b577-c0b5-4f55-aa34-99328b749f3e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:25.974447 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1405/2810] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:13:25 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:13:25.979488 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4552ab-990a-4afa-9a08-040e44ad44c6 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:25.990424 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4552ab-990a-4afa-9a08-040e44ad44c6 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:25.995101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b8b58-666f-4044-a88b-08c2f202e19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:25.995752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b8b58-666f-4044-a88b-08c2f202e19c admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:25.996037 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b8b58-666f-4044-a88b-08c2f202e19c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:25.996312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b8b58-666f-4044-a88b-08c2f202e19c admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:25.997092 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b8b58-666f-4044-a88b-08c2f202e19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:25.998688 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b8b58-666f-4044-a88b-08c2f202e19c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:26.009999 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1406/2811] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:13:26.014003 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769b577-c0b5-4f55-aa34-99328b749f3e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:26.024909 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769b577-c0b5-4f55-aa34-99328b749f3e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:26.031864 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33177e3a-16ce-4ecb-b312-51d888484af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:26.032877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33177e3a-16ce-4ecb-b312-51d888484af8 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:26.033324 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33177e3a-16ce-4ecb-b312-51d888484af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:26.033766 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33177e3a-16ce-4ecb-b312-51d888484af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:26.034960 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33177e3a-16ce-4ecb-b312-51d888484af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:26.037581 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33177e3a-16ce-4ecb-b312-51d888484af8 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:26.053693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1406/2812] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:13:26.058462 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b8b58-666f-4044-a88b-08c2f202e19c admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:26.069330 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b8b58-666f-4044-a88b-08c2f202e19c admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:26.075188 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbfd10-afd0-4366-aef5-ca306f3fd138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:26.076101 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbfd10-afd0-4366-aef5-ca306f3fd138 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:26.076426 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbfd10-afd0-4366-aef5-ca306f3fd138 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:26.076745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbfd10-afd0-4366-aef5-ca306f3fd138 admin admin] PATH_INFO: `/v3/projects/83fc516bfc2e4d8ba523091299e244ab/users/409d564d70cf4fa59813e2c894a78341/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:26.093090 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbfd10-afd0-4366-aef5-ca306f3fd138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fc516bfc2e4d8ba523091299e244ab, user_id=409d564d70cf4fa59813e2c894a78341, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:26.095745 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbfd10-afd0-4366-aef5-ca306f3fd138 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:26.109399 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1407/2813] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:13:26 2022] PUT /identity/v3/projects/83fc516bfc2e4d8ba523091299e244ab/users/409d564d70cf4fa59813e2c894a78341/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:26.121857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a249b-cc5e-456e-a397-de163fcf0db7 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:26.122142 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a249b-cc5e-456e-a397-de163fcf0db7 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:26.122419 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a249b-cc5e-456e-a397-de163fcf0db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:26.129226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:26.154399 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-922a249b-cc5e-456e-a397-de163fcf0db7 None None] MFA Rules not processed for user `409d564d70cf4fa59813e2c894a78341`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:26.155637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-922a249b-cc5e-456e-a397-de163fcf0db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:26.223536 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1407/2814] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:26.228717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbfd10-afd0-4366-aef5-ca306f3fd138 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:26.291936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbfd10-afd0-4366-aef5-ca306f3fd138 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:26.296693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b45d17-4c5c-47f1-95a2-8231c2a6bfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:26.297349 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b45d17-4c5c-47f1-95a2-8231c2a6bfcb admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:26.297638 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b45d17-4c5c-47f1-95a2-8231c2a6bfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:26.297912 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b45d17-4c5c-47f1-95a2-8231c2a6bfcb admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:26.298606 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b45d17-4c5c-47f1-95a2-8231c2a6bfcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:26.300878 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b45d17-4c5c-47f1-95a2-8231c2a6bfcb admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:26.330052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1408/2815] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:13:26 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:13:26.334192 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a249b-cc5e-456e-a397-de163fcf0db7 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:26.345385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a249b-cc5e-456e-a397-de163fcf0db7 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:26.405431 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b40be-176e-44c5-a19b-7f45f89180e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:26.406109 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b40be-176e-44c5-a19b-7f45f89180e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:26.406394 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b40be-176e-44c5-a19b-7f45f89180e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:26.406667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b40be-176e-44c5-a19b-7f45f89180e4 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:26.407358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b40be-176e-44c5-a19b-7f45f89180e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:26.409309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b40be-176e-44c5-a19b-7f45f89180e4 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:26.430137 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1408/2816] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:13:26 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:13:26.434321 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b45d17-4c5c-47f1-95a2-8231c2a6bfcb admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:26.446018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b45d17-4c5c-47f1-95a2-8231c2a6bfcb admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:26.451342 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f12f7-ead0-45ee-a349-864f2899910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:26.452005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f12f7-ead0-45ee-a349-864f2899910e admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:26.452288 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f12f7-ead0-45ee-a349-864f2899910e admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:26.452749 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f12f7-ead0-45ee-a349-864f2899910e admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:26.453476 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f12f7-ead0-45ee-a349-864f2899910e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:26.455057 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f12f7-ead0-45ee-a349-864f2899910e admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:26.466508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1409/2817] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:13:26.471319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b40be-176e-44c5-a19b-7f45f89180e4 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:26.481673 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b40be-176e-44c5-a19b-7f45f89180e4 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:26.486400 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a50426-81d4-433f-91d2-90134f9a92f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:26.487274 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50426-81d4-433f-91d2-90134f9a92f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:26.487557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50426-81d4-433f-91d2-90134f9a92f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:26.487834 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50426-81d4-433f-91d2-90134f9a92f3 admin admin] PATH_INFO: `/v3/projects/a9bc115e045f4cddaf87e29acdf71272/users/e7e7aada540e4227b837c30796a7489d/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:26.503112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a50426-81d4-433f-91d2-90134f9a92f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9bc115e045f4cddaf87e29acdf71272, user_id=e7e7aada540e4227b837c30796a7489d, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:26.505779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a50426-81d4-433f-91d2-90134f9a92f3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:26.519313 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1409/2818] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:13:26 2022] PUT /identity/v3/projects/a9bc115e045f4cddaf87e29acdf71272/users/e7e7aada540e4227b837c30796a7489d/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:26.523469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f12f7-ead0-45ee-a349-864f2899910e admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:26.534659 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f12f7-ead0-45ee-a349-864f2899910e admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:26.577736 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:26.595857 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73972e-ba46-477c-8934-9f6edcab2f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:26.596556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73972e-ba46-477c-8934-9f6edcab2f4c admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:26.596841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73972e-ba46-477c-8934-9f6edcab2f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:26.597116 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73972e-ba46-477c-8934-9f6edcab2f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:26.597868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73972e-ba46-477c-8934-9f6edcab2f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:26.599447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73972e-ba46-477c-8934-9f6edcab2f4c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:26.610043 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1410/2819] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:13:26.614820 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a50426-81d4-433f-91d2-90134f9a92f3 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:26.626059 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a50426-81d4-433f-91d2-90134f9a92f3 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:26.631000 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf47e4-f88b-4e8f-800f-e929db266d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:26.631872 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf47e4-f88b-4e8f-800f-e929db266d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:26.632155 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf47e4-f88b-4e8f-800f-e929db266d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:26.632497 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf47e4-f88b-4e8f-800f-e929db266d32 admin admin] PATH_INFO: `/v3/projects/a9bc115e045f4cddaf87e29acdf71272/users/e7e7aada540e4227b837c30796a7489d/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:26.635096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf47e4-f88b-4e8f-800f-e929db266d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9bc115e045f4cddaf87e29acdf71272, user_id=e7e7aada540e4227b837c30796a7489d, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:26.637783 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf47e4-f88b-4e8f-800f-e929db266d32 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:26.648006 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1410/2820] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:13:26 2022] PUT /identity/v3/projects/a9bc115e045f4cddaf87e29acdf71272/users/e7e7aada540e4227b837c30796a7489d/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:26.658568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bc473-25a0-4314-80af-ecaaa92e95ec None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:26.658909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bc473-25a0-4314-80af-ecaaa92e95ec None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:26.659451 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bc473-25a0-4314-80af-ecaaa92e95ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:26.687471 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-8e8bc473-25a0-4314-80af-ecaaa92e95ec None None] MFA Rules not processed for user `e7e7aada540e4227b837c30796a7489d`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:26.688717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8e8bc473-25a0-4314-80af-ecaaa92e95ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:26.759893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1411/2821] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:26.772497 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf47e4-f88b-4e8f-800f-e929db266d32 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:26.774396 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-53bf47e4-f88b-4e8f-800f-e929db266d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:26.822165 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:26.834667 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf47e4-f88b-4e8f-800f-e929db266d32 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:26.838249 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50683c07-4e64-4276-85dc-ba166b64afa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:26.838896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50683c07-4e64-4276-85dc-ba166b64afa6 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:26.839182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50683c07-4e64-4276-85dc-ba166b64afa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:26.839461 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50683c07-4e64-4276-85dc-ba166b64afa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:26.848738 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50683c07-4e64-4276-85dc-ba166b64afa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:26.850160 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50683c07-4e64-4276-85dc-ba166b64afa6 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:26.860226 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1411/2822] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:13:27.165316 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8bc473-25a0-4314-80af-ecaaa92e95ec None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:27.174807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8bc473-25a0-4314-80af-ecaaa92e95ec None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:27.179544 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1866159-96f5-4fde-8643-3614a522ea49 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7e7aada540e4227b837c30796a7489d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9bc115e045f4cddaf87e29acdf71272', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:27.180779 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1866159-96f5-4fde-8643-3614a522ea49 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:27.181077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1866159-96f5-4fde-8643-3614a522ea49 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:27.181361 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1866159-96f5-4fde-8643-3614a522ea49 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:27.182086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1866159-96f5-4fde-8643-3614a522ea49 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:13:27.182396 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1866159-96f5-4fde-8643-3614a522ea49 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:27.183991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1866159-96f5-4fde-8643-3614a522ea49 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:27.195698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1412/2823] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:13:27.200185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50683c07-4e64-4276-85dc-ba166b64afa6 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:27.209725 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50683c07-4e64-4276-85dc-ba166b64afa6 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:27.214899 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5daa4-b253-4437-bb02-af60494a647d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7e7aada540e4227b837c30796a7489d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9bc115e045f4cddaf87e29acdf71272', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:27.215560 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5daa4-b253-4437-bb02-af60494a647d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:27.215844 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5daa4-b253-4437-bb02-af60494a647d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:27.216115 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5daa4-b253-4437-bb02-af60494a647d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:27.216842 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5daa4-b253-4437-bb02-af60494a647d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:27.218732 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5daa4-b253-4437-bb02-af60494a647d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:27.245019 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1412/2824] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:13:27 2022] POST /identity/v3/projects => generated 354 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:13:27.249319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1866159-96f5-4fde-8643-3614a522ea49 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:27.259581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1866159-96f5-4fde-8643-3614a522ea49 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:27.317331 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad05e7-9a56-4705-87d1-aa49fe193f85 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7e7aada540e4227b837c30796a7489d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9bc115e045f4cddaf87e29acdf71272', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:27.318007 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad05e7-9a56-4705-87d1-aa49fe193f85 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:27.318298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad05e7-9a56-4705-87d1-aa49fe193f85 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:27.318583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad05e7-9a56-4705-87d1-aa49fe193f85 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:27.319310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad05e7-9a56-4705-87d1-aa49fe193f85 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:13:27.319612 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad05e7-9a56-4705-87d1-aa49fe193f85 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:27.321237 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad05e7-9a56-4705-87d1-aa49fe193f85 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:27.333707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1413/2825] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:13:27.370169 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5daa4-b253-4437-bb02-af60494a647d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:27.380923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5daa4-b253-4437-bb02-af60494a647d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:27.385678 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7e7aada540e4227b837c30796a7489d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9bc115e045f4cddaf87e29acdf71272', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:27.386616 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:27.387017 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:27.387229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] PATH_INFO: `/v3/projects/5bb637de1a9f46d5a49ac3687fee91a7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:27.388604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5bb637de1a9f46d5a49ac3687fee91a7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:27.390716 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:27.438213 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb637de1a9f46d5a49ac3687fee91a7', 'request_id': 'req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:27.446071 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb637de1a9f46d5a49ac3687fee91a7', 'request_id': 'req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:27.453177 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb637de1a9f46d5a49ac3687fee91a7', 'request_id': 'req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:27.463916 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] The token cache is being invalidate because project 5bb637de1a9f46d5a49ac3687fee91a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:13:27.464203 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:27.464901 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-60c6a9c3-8811-4ce9-bc5b-c429b641b56d tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:27.466857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1413/2826] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:13:27 2022] DELETE /identity/v3/projects/5bb637de1a9f46d5a49ac3687fee91a7 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:27.674284 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad05e7-9a56-4705-87d1-aa49fe193f85 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:27.676036 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-84ad05e7-9a56-4705-87d1-aa49fe193f85 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:27.726516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=100 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:27.745834 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad05e7-9a56-4705-87d1-aa49fe193f85 tempest-VolumeQuotaClassesTest-1335380647 tempest-VolumeQuotaClassesTest-1335380647-project-admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:27.801981 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:27.802856 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:27.803142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:27.803421 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] PATH_INFO: `/v3/users/409d564d70cf4fa59813e2c894a78341` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:27.804765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=409d564d70cf4fa59813e2c894a78341)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:27.807298 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:27.858865 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409d564d70cf4fa59813e2c894a78341'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:27.873189 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409d564d70cf4fa59813e2c894a78341'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:27.874019 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409d564d70cf4fa59813e2c894a78341'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:27.887470 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409d564d70cf4fa59813e2c894a78341'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:27.897051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409d564d70cf4fa59813e2c894a78341'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:27.904175 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1414/2827] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:13:27 2022] DELETE /identity/v3/users/409d564d70cf4fa59813e2c894a78341 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:27.909138 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038f9b9-be7e-4054-b777-fb94dfabfd7d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:27.909623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038f9b9-be7e-4054-b777-fb94dfabfd7d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:27.909623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038f9b9-be7e-4054-b777-fb94dfabfd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:27.927312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-0038f9b9-be7e-4054-b777-fb94dfabfd7d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:27.928662 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0038f9b9-be7e-4054-b777-fb94dfabfd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:27.989350 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1414/2828] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:13:28.003053 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:28.004803 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:28.064960 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013e272-c5d1-4f79-8e52-44ec267092e9 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:28.068710 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3145ec3-6485-40f7-9c5c-92e50462b968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:28.069362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3145ec3-6485-40f7-9c5c-92e50462b968 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:28.069649 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3145ec3-6485-40f7-9c5c-92e50462b968 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:28.069930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3145ec3-6485-40f7-9c5c-92e50462b968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:28.080224 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3145ec3-6485-40f7-9c5c-92e50462b968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:28.081697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3145ec3-6485-40f7-9c5c-92e50462b968 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:28.091270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1415/2829] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:13:28.365443 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038f9b9-be7e-4054-b777-fb94dfabfd7d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:28.366828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-0038f9b9-be7e-4054-b777-fb94dfabfd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:28.379900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038f9b9-be7e-4054-b777-fb94dfabfd7d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:28.384689 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:28.385549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:28.385839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:28.386116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] PATH_INFO: `/v3/projects/83fc516bfc2e4d8ba523091299e244ab` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:28.387344 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83fc516bfc2e4d8ba523091299e244ab)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:28.389299 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:28.435672 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:28.439679 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83fc516bfc2e4d8ba523091299e244ab', 'request_id': 'req-b7f78981-625b-40ba-9161-c4e868b3b860'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:28.447675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83fc516bfc2e4d8ba523091299e244ab', 'request_id': 'req-b7f78981-625b-40ba-9161-c4e868b3b860'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:28.454694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83fc516bfc2e4d8ba523091299e244ab', 'request_id': 'req-b7f78981-625b-40ba-9161-c4e868b3b860'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:28.465306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] The token cache is being invalidate because project 83fc516bfc2e4d8ba523091299e244ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:13:28.465597 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:28.466264 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:28.467828 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1415/2830] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:13:28 2022] DELETE /identity/v3/projects/83fc516bfc2e4d8ba523091299e244ab => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:28.472016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3145ec3-6485-40f7-9c5c-92e50462b968 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:28.482279 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3145ec3-6485-40f7-9c5c-92e50462b968 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:28.491184 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:28.542385 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2211f-a48a-4cfd-ad25-f6358da5b107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:28.543257 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2211f-a48a-4cfd-ad25-f6358da5b107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:28.543545 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2211f-a48a-4cfd-ad25-f6358da5b107 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:28.543825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2211f-a48a-4cfd-ad25-f6358da5b107 admin admin] PATH_INFO: `/v3/users/e7e7aada540e4227b837c30796a7489d` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:28.545040 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2211f-a48a-4cfd-ad25-f6358da5b107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e7aada540e4227b837c30796a7489d)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:28.547109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2211f-a48a-4cfd-ad25-f6358da5b107 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:28.593596 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-8ca2211f-a48a-4cfd-ad25-f6358da5b107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e7aada540e4227b837c30796a7489d'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:28.601868 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-8ca2211f-a48a-4cfd-ad25-f6358da5b107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e7aada540e4227b837c30796a7489d'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:28.602527 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-8ca2211f-a48a-4cfd-ad25-f6358da5b107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e7aada540e4227b837c30796a7489d'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:28.614841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-8ca2211f-a48a-4cfd-ad25-f6358da5b107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e7aada540e4227b837c30796a7489d'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:28.624306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-8ca2211f-a48a-4cfd-ad25-f6358da5b107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e7aada540e4227b837c30796a7489d'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:28.630746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1416/2831] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:13:28 2022] DELETE /identity/v3/users/e7e7aada540e4227b837c30796a7489d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:28.904184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:28.905685 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:28.922437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f78981-625b-40ba-9161-c4e868b3b860 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:28.984873 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:28.985653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:28.985938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:28.986216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] PATH_INFO: `/v3/projects/a9bc115e045f4cddaf87e29acdf71272` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:28.987404 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9bc115e045f4cddaf87e29acdf71272)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:28.989333 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:29.035407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9bc115e045f4cddaf87e29acdf71272', 'request_id': 'req-afd1c993-9e45-4b36-805f-eacf7479dc75'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:29.043337 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9bc115e045f4cddaf87e29acdf71272', 'request_id': 'req-afd1c993-9e45-4b36-805f-eacf7479dc75'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:29.050686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9bc115e045f4cddaf87e29acdf71272', 'request_id': 'req-afd1c993-9e45-4b36-805f-eacf7479dc75'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:29.060849 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] The token cache is being invalidate because project a9bc115e045f4cddaf87e29acdf71272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:13:29.061133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:29.061770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-afd1c993-9e45-4b36-805f-eacf7479dc75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:29.063296 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1416/2832] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:13:28 2022] DELETE /identity/v3/projects/a9bc115e045f4cddaf87e29acdf71272 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:29.068442 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5005b-6e29-4a81-8c75-b49ff98ae6b5 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.068926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5005b-6e29-4a81-8c75-b49ff98ae6b5 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.069050 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5005b-6e29-4a81-8c75-b49ff98ae6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.088425 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-b0e5005b-6e29-4a81-8c75-b49ff98ae6b5 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:29.089640 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-b0e5005b-6e29-4a81-8c75-b49ff98ae6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:29.149810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1417/2833] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:29.157849 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b926da-e9f9-44bc-82c1-d6830b8baea6 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.158133 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b926da-e9f9-44bc-82c1-d6830b8baea6 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.158408 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b926da-e9f9-44bc-82c1-d6830b8baea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.176712 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e5b926da-e9f9-44bc-82c1-d6830b8baea6 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:29.177923 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e5b926da-e9f9-44bc-82c1-d6830b8baea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:29.186028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1417/2834] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:29.190455 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5005b-6e29-4a81-8c75-b49ff98ae6b5 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:29.204550 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5005b-6e29-4a81-8c75-b49ff98ae6b5 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:29.211807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9fd7b-8e2d-41a6-9bff-4aa37c3b509a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:29.212820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9fd7b-8e2d-41a6-9bff-4aa37c3b509a admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.213252 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9fd7b-8e2d-41a6-9bff-4aa37c3b509a admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.213698 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9fd7b-8e2d-41a6-9bff-4aa37c3b509a admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.214823 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9fd7b-8e2d-41a6-9bff-4aa37c3b509a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:13:29.215289 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9fd7b-8e2d-41a6-9bff-4aa37c3b509a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:29.217814 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9fd7b-8e2d-41a6-9bff-4aa37c3b509a admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:29.236103 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1418/2835] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:13:29.241779 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b926da-e9f9-44bc-82c1-d6830b8baea6 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:29.251680 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b926da-e9f9-44bc-82c1-d6830b8baea6 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:29.256530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5752eb5-5f6e-4086-8424-908c6176f045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:29.257184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5752eb5-5f6e-4086-8424-908c6176f045 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.257463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5752eb5-5f6e-4086-8424-908c6176f045 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.258229 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5752eb5-5f6e-4086-8424-908c6176f045 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.258925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5752eb5-5f6e-4086-8424-908c6176f045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:29.260858 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5752eb5-5f6e-4086-8424-908c6176f045 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:29.287641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1418/2836] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:13:29 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:13:29.292004 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9fd7b-8e2d-41a6-9bff-4aa37c3b509a admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:29.302157 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9fd7b-8e2d-41a6-9bff-4aa37c3b509a admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:29.319135 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:29.361023 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f1104-cf05-4a91-995d-9d90b9429314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:29.361697 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f1104-cf05-4a91-995d-9d90b9429314 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.361987 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f1104-cf05-4a91-995d-9d90b9429314 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.362270 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f1104-cf05-4a91-995d-9d90b9429314 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.362967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f1104-cf05-4a91-995d-9d90b9429314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:29.364908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f1104-cf05-4a91-995d-9d90b9429314 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:29.385125 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1419/2837] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:13:29 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:13:29.389451 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5752eb5-5f6e-4086-8424-908c6176f045 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:29.400801 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5752eb5-5f6e-4086-8424-908c6176f045 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:29.405489 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad14afa-10a6-4e0c-9065-f8564a726330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:29.406150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad14afa-10a6-4e0c-9065-f8564a726330 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.406436 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad14afa-10a6-4e0c-9065-f8564a726330 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.406720 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad14afa-10a6-4e0c-9065-f8564a726330 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.407463 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad14afa-10a6-4e0c-9065-f8564a726330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:29.409101 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad14afa-10a6-4e0c-9065-f8564a726330 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:29.419913 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1419/2838] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:13:29.424164 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f1104-cf05-4a91-995d-9d90b9429314 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:29.435221 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f1104-cf05-4a91-995d-9d90b9429314 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:29.440077 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cef59-0e29-4e6e-8135-d044c176fa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:29.440808 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cef59-0e29-4e6e-8135-d044c176fa21 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.441099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cef59-0e29-4e6e-8135-d044c176fa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.441383 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cef59-0e29-4e6e-8135-d044c176fa21 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.442121 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2cef59-0e29-4e6e-8135-d044c176fa21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:29.443757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2cef59-0e29-4e6e-8135-d044c176fa21 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:29.455384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1420/2839] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:13:29.459393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad14afa-10a6-4e0c-9065-f8564a726330 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:29.470130 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad14afa-10a6-4e0c-9065-f8564a726330 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:29.474887 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a819d8-2beb-4932-9cf1-f4b189e5e3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:29.475763 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a819d8-2beb-4932-9cf1-f4b189e5e3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.476050 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a819d8-2beb-4932-9cf1-f4b189e5e3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.476751 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a819d8-2beb-4932-9cf1-f4b189e5e3f8 admin admin] PATH_INFO: `/v3/projects/54a52cc7080a4d22b674d41f730339f0/users/316491a554b54fcfb239a99667532bed/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.490914 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a819d8-2beb-4932-9cf1-f4b189e5e3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a52cc7080a4d22b674d41f730339f0, user_id=316491a554b54fcfb239a99667532bed, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:29.494097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a819d8-2beb-4932-9cf1-f4b189e5e3f8 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:29.503175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:29.510150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1420/2840] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:13:29 2022] PUT /identity/v3/projects/54a52cc7080a4d22b674d41f730339f0/users/316491a554b54fcfb239a99667532bed/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:29.521337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d2390-2dee-43b1-a8e8-0e87a6b1ddd0 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.521628 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d2390-2dee-43b1-a8e8-0e87a6b1ddd0 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.521937 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d2390-2dee-43b1-a8e8-0e87a6b1ddd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.550110 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-911d2390-2dee-43b1-a8e8-0e87a6b1ddd0 None None] MFA Rules not processed for user `316491a554b54fcfb239a99667532bed`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:29.551322 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-911d2390-2dee-43b1-a8e8-0e87a6b1ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:29.618307 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1421/2841] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:29.629416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a819d8-2beb-4932-9cf1-f4b189e5e3f8 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:29.631097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d7a819d8-2beb-4932-9cf1-f4b189e5e3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:29.690908 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a819d8-2beb-4932-9cf1-f4b189e5e3f8 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:29.694616 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449692f-f5cc-4564-bbe4-59a58046ca32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:29.695810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449692f-f5cc-4564-bbe4-59a58046ca32 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.696129 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449692f-f5cc-4564-bbe4-59a58046ca32 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.696431 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449692f-f5cc-4564-bbe4-59a58046ca32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.705744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449692f-f5cc-4564-bbe4-59a58046ca32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:29.707175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449692f-f5cc-4564-bbe4-59a58046ca32 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:29.716648 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1421/2842] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:13:29.739421 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d2390-2dee-43b1-a8e8-0e87a6b1ddd0 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:29.807588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d2390-2dee-43b1-a8e8-0e87a6b1ddd0 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:29.812512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:29.813525 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.813884 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.814067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] PATH_INFO: `/v3/users/316491a554b54fcfb239a99667532bed` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.815343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] RBAC: Authorizing `identity:delete_user(user_id=316491a554b54fcfb239a99667532bed)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:29.817360 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:29.854035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:29.873597 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '316491a554b54fcfb239a99667532bed'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:29.881508 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '316491a554b54fcfb239a99667532bed'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:29.882277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '316491a554b54fcfb239a99667532bed'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:29.895611 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '316491a554b54fcfb239a99667532bed'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:29.903554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '316491a554b54fcfb239a99667532bed'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:29.910176 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1422/2843] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:13:29 2022] DELETE /identity/v3/users/316491a554b54fcfb239a99667532bed => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:29.915561 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceea564-f7a6-4d14-a43c-0c065f34cdcf None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:29.915881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceea564-f7a6-4d14-a43c-0c065f34cdcf None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:29.916163 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceea564-f7a6-4d14-a43c-0c065f34cdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:29.935989 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-fceea564-f7a6-4d14-a43c-0c065f34cdcf None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:29.937254 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fceea564-f7a6-4d14-a43c-0c065f34cdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:29.998271 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1422/2844] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:30.008095 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:30.009699 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:30.071879 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4affd7-df4c-4a52-bf60-cdeddd88c27b admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:30.076223 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb486a-7d97-46f5-8d2c-269613fd0031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:30.076913 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb486a-7d97-46f5-8d2c-269613fd0031 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:30.077203 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb486a-7d97-46f5-8d2c-269613fd0031 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:30.077481 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb486a-7d97-46f5-8d2c-269613fd0031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:30.087397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb486a-7d97-46f5-8d2c-269613fd0031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:30.088880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb486a-7d97-46f5-8d2c-269613fd0031 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:30.098469 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1423/2845] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:13:30.372709 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceea564-f7a6-4d14-a43c-0c065f34cdcf None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:30.374154 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fceea564-f7a6-4d14-a43c-0c065f34cdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:30.386455 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceea564-f7a6-4d14-a43c-0c065f34cdcf None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:30.391374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:30.392245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:30.392594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:30.392875 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] PATH_INFO: `/v3/projects/54a52cc7080a4d22b674d41f730339f0` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:30.394104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54a52cc7080a4d22b674d41f730339f0)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:30.396037 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:30.444494 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54a52cc7080a4d22b674d41f730339f0', 'request_id': 'req-eccffb14-2741-4471-b535-3c7f9833c3f7'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:30.452444 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54a52cc7080a4d22b674d41f730339f0', 'request_id': 'req-eccffb14-2741-4471-b535-3c7f9833c3f7'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:30.458883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54a52cc7080a4d22b674d41f730339f0', 'request_id': 'req-eccffb14-2741-4471-b535-3c7f9833c3f7'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:30.469718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] The token cache is being invalidate because project 54a52cc7080a4d22b674d41f730339f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86669) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:13:30.470015 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:30.470660 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-eccffb14-2741-4471-b535-3c7f9833c3f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:13:30.472202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1423/2846] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:13:30 2022] DELETE /identity/v3/projects/54a52cc7080a4d22b674d41f730339f0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:13:30.477682 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f6279-7dcb-4e67-abf6-e5c2320114d3 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:30.477975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f6279-7dcb-4e67-abf6-e5c2320114d3 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:30.478258 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f6279-7dcb-4e67-abf6-e5c2320114d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:30.496201 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-3c1f6279-7dcb-4e67-abf6-e5c2320114d3 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:30.497446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3c1f6279-7dcb-4e67-abf6-e5c2320114d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:30.561858 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1424/2847] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:30.569654 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9beed7-ae3e-44ad-8103-56bf5abd0bb5 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:30.569939 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9beed7-ae3e-44ad-8103-56bf5abd0bb5 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:30.570821 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9beed7-ae3e-44ad-8103-56bf5abd0bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:30.589165 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-ca9beed7-ae3e-44ad-8103-56bf5abd0bb5 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:30.590363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-ca9beed7-ae3e-44ad-8103-56bf5abd0bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:30.598234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1424/2848] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:30.602932 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f6279-7dcb-4e67-abf6-e5c2320114d3 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:30.613404 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f6279-7dcb-4e67-abf6-e5c2320114d3 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:30.618821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b8eb3-44ab-458a-b835-ceb9a2e2dfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:30.619427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b8eb3-44ab-458a-b835-ceb9a2e2dfdb admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:30.619718 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b8eb3-44ab-458a-b835-ceb9a2e2dfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:30.619996 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b8eb3-44ab-458a-b835-ceb9a2e2dfdb admin admin] PATH_INFO: `/v3/domains` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:30.620758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b8eb3-44ab-458a-b835-ceb9a2e2dfdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86668) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:13:30.621066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b8eb3-44ab-458a-b835-ceb9a2e2dfdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:30.622660 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b8eb3-44ab-458a-b835-ceb9a2e2dfdb admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:30.634647 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1425/2849] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:13:30.639189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9beed7-ae3e-44ad-8103-56bf5abd0bb5 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:30.648476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9beed7-ae3e-44ad-8103-56bf5abd0bb5 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:30.653165 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd820282-c1f8-4bcd-b73a-59a84e6dba39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:30.653809 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd820282-c1f8-4bcd-b73a-59a84e6dba39 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:30.654089 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd820282-c1f8-4bcd-b73a-59a84e6dba39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:30.654361 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd820282-c1f8-4bcd-b73a-59a84e6dba39 admin admin] PATH_INFO: `/v3/projects` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:30.655125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd820282-c1f8-4bcd-b73a-59a84e6dba39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:30.657048 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd820282-c1f8-4bcd-b73a-59a84e6dba39 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:30.671608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:30.687184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1425/2850] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:13:30 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:13:30.691397 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b8eb3-44ab-458a-b835-ceb9a2e2dfdb admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:30.702005 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b8eb3-44ab-458a-b835-ceb9a2e2dfdb admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:30.736202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:30.761893 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02d10b-9203-4f80-a1f5-6a4e054532a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:30.762607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d10b-9203-4f80-a1f5-6a4e054532a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:30.762895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d10b-9203-4f80-a1f5-6a4e054532a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:30.763179 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d10b-9203-4f80-a1f5-6a4e054532a4 admin admin] PATH_INFO: `/v3/users` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:30.763873 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02d10b-9203-4f80-a1f5-6a4e054532a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:30.765883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02d10b-9203-4f80-a1f5-6a4e054532a4 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:30.785936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1426/2851] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:13:30 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:13:30.790623 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd820282-c1f8-4bcd-b73a-59a84e6dba39 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:30.801386 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd820282-c1f8-4bcd-b73a-59a84e6dba39 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:30.806812 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985c248-f8dc-4250-995d-5abe675f8c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:30.807468 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985c248-f8dc-4250-995d-5abe675f8c83 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:30.807749 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985c248-f8dc-4250-995d-5abe675f8c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:30.808055 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985c248-f8dc-4250-995d-5abe675f8c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:30.808808 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985c248-f8dc-4250-995d-5abe675f8c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:30.810403 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985c248-f8dc-4250-995d-5abe675f8c83 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:30.821227 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1426/2852] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:13:30.825670 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02d10b-9203-4f80-a1f5-6a4e054532a4 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:30.835895 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02d10b-9203-4f80-a1f5-6a4e054532a4 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:30.840583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab75a5-655d-4953-b019-0dcaf9096105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:30.841242 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab75a5-655d-4953-b019-0dcaf9096105 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:30.841523 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab75a5-655d-4953-b019-0dcaf9096105 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:30.841799 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab75a5-655d-4953-b019-0dcaf9096105 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:30.842553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab75a5-655d-4953-b019-0dcaf9096105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:30.844136 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab75a5-655d-4953-b019-0dcaf9096105 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:30.854679 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1427/2853] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:13:30.858822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985c248-f8dc-4250-995d-5abe675f8c83 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:30.869448 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985c248-f8dc-4250-995d-5abe675f8c83 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:30.874187 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6b000-045a-48d7-a87b-f035a5670a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:30.875066 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6b000-045a-48d7-a87b-f035a5670a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:30.875345 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6b000-045a-48d7-a87b-f035a5670a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:30.875619 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6b000-045a-48d7-a87b-f035a5670a27 admin admin] PATH_INFO: `/v3/projects/8bf42d21f274403fbf00d93bff8b263e/users/ea9a3847de1147a09506b79496ccece9/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:30.890393 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6b000-045a-48d7-a87b-f035a5670a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf42d21f274403fbf00d93bff8b263e, user_id=ea9a3847de1147a09506b79496ccece9, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:30.893218 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6b000-045a-48d7-a87b-f035a5670a27 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:30.909168 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1427/2854] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:13:30 2022] PUT /identity/v3/projects/8bf42d21f274403fbf00d93bff8b263e/users/ea9a3847de1147a09506b79496ccece9/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:13:30.921415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af3ae4-ed05-45ac-af9f-82eafec55b6c None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:30.921730 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af3ae4-ed05-45ac-af9f-82eafec55b6c None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:30.922017 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af3ae4-ed05-45ac-af9f-82eafec55b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:30.950599 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-d2af3ae4-ed05-45ac-af9f-82eafec55b6c None None] MFA Rules not processed for user `ea9a3847de1147a09506b79496ccece9`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:13:30.951839 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d2af3ae4-ed05-45ac-af9f-82eafec55b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:31.015717 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1428/2855] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:13:31.027934 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6b000-045a-48d7-a87b-f035a5670a27 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:31.029999 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-7db6b000-045a-48d7-a87b-f035a5670a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:31.087391 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6b000-045a-48d7-a87b-f035a5670a27 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:31.091302 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031213bb-3e9e-4919-84a7-5b6d4324ce60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35e7720e3bf4c9e80bdefa6a1cb8296', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:31.091961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031213bb-3e9e-4919-84a7-5b6d4324ce60 service glance] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:31.092245 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031213bb-3e9e-4919-84a7-5b6d4324ce60 service glance] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:31.092546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031213bb-3e9e-4919-84a7-5b6d4324ce60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:31.103018 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031213bb-3e9e-4919-84a7-5b6d4324ce60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:31.104525 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031213bb-3e9e-4919-84a7-5b6d4324ce60 service glance] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:31.118977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1428/2856] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:13:35.545362 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af3ae4-ed05-45ac-af9f-82eafec55b6c None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:13:35.546825 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d2af3ae4-ed05-45ac-af9f-82eafec55b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:35.629427 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af3ae4-ed05-45ac-af9f-82eafec55b6c None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:13:35.635583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34ec32-74b5-4c2a-ac94-fe4f1e53f5dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:13:35.636291 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ec32-74b5-4c2a-ac94-fe4f1e53f5dc service placement] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:13:35.636609 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ec32-74b5-4c2a-ac94-fe4f1e53f5dc service placement] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:13:35.636971 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ec32-74b5-4c2a-ac94-fe4f1e53f5dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:13:35.638737 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0a34ec32-74b5-4c2a-ac94-fe4f1e53f5dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:13:35.651300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34ec32-74b5-4c2a-ac94-fe4f1e53f5dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:13:35.653075 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34ec32-74b5-4c2a-ac94-fe4f1e53f5dc service placement] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:13:35.663751 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1429/2857] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:13:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:13:41.109836 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:13:45.659182 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:14:14.019230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031213bb-3e9e-4919-84a7-5b6d4324ce60 service glance] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:14.083807 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031213bb-3e9e-4919-84a7-5b6d4324ce60 service glance] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:14.088415 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:14.089230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:14.089509 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:14.089784 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] PATH_INFO: `/v3/users/ea9a3847de1147a09506b79496ccece9` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:14.091594 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9a3847de1147a09506b79496ccece9)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:14.093516 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:14.143329 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9a3847de1147a09506b79496ccece9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:14.151240 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9a3847de1147a09506b79496ccece9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:14.159162 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9a3847de1147a09506b79496ccece9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:14.159851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9a3847de1147a09506b79496ccece9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:14.171700 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9a3847de1147a09506b79496ccece9'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:14.177671 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1429/2858] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:14:14 2022] DELETE /identity/v3/users/ea9a3847de1147a09506b79496ccece9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:14:14.182509 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d39a12-8ad9-42ae-b7e9-d016f1aea2e5 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:14.183044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d39a12-8ad9-42ae-b7e9-d016f1aea2e5 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:14.183044 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d39a12-8ad9-42ae-b7e9-d016f1aea2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:14.203391 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-f3d39a12-8ad9-42ae-b7e9-d016f1aea2e5 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:14.204693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f3d39a12-8ad9-42ae-b7e9-d016f1aea2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:14.268238 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1430/2859] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:14.279551 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:14.281008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:14.343892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e39837-9720-48f0-a28c-3526d01f787e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:14.347593 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956ab47-585a-4159-baf3-52416a3627a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:14.348237 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956ab47-585a-4159-baf3-52416a3627a9 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:14.348546 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956ab47-585a-4159-baf3-52416a3627a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:14.348822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956ab47-585a-4159-baf3-52416a3627a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:14.358776 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956ab47-585a-4159-baf3-52416a3627a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:14.360207 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956ab47-585a-4159-baf3-52416a3627a9 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:14.370358 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1430/2860] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:14:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:14:14.642938 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d39a12-8ad9-42ae-b7e9-d016f1aea2e5 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:14.644585 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-f3d39a12-8ad9-42ae-b7e9-d016f1aea2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:14.657458 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d39a12-8ad9-42ae-b7e9-d016f1aea2e5 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:14.662378 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:14.663272 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:14.663583 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:14.663861 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] PATH_INFO: `/v3/projects/8bf42d21f274403fbf00d93bff8b263e` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:14.665186 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf42d21f274403fbf00d93bff8b263e)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:14.667038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:14.715056 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf42d21f274403fbf00d93bff8b263e', 'request_id': 'req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:14.722236 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf42d21f274403fbf00d93bff8b263e', 'request_id': 'req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:14.732560 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf42d21f274403fbf00d93bff8b263e', 'request_id': 'req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:14.743158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] The token cache is being invalidate because project 8bf42d21f274403fbf00d93bff8b263e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:14:14.743446 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:14.744099 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-2ed45fe6-f0a4-43a3-812f-c9427a3ab0f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:14.745664 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1431/2861] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:14:14 2022] DELETE /identity/v3/projects/8bf42d21f274403fbf00d93bff8b263e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:14:14.751739 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c04402-d4f3-4923-8ba4-148abd7410ca None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:14.752034 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c04402-d4f3-4923-8ba4-148abd7410ca None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:14.752319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c04402-d4f3-4923-8ba4-148abd7410ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:14.770576 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-a3c04402-d4f3-4923-8ba4-148abd7410ca None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:14.771704 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a3c04402-d4f3-4923-8ba4-148abd7410ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:14.790070 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:14:14.843367 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1431/2862] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:14.851894 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db680f-a2d5-4514-ab31-c2b1519b25c7 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:14.852211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db680f-a2d5-4514-ab31-c2b1519b25c7 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:14.852690 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db680f-a2d5-4514-ab31-c2b1519b25c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:14.873542 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-30db680f-a2d5-4514-ab31-c2b1519b25c7 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:14.874775 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-30db680f-a2d5-4514-ab31-c2b1519b25c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:14.883096 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1432/2863] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:14.887637 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c04402-d4f3-4923-8ba4-148abd7410ca None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:14.897360 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c04402-d4f3-4923-8ba4-148abd7410ca None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:14.901983 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932ef20-b1fb-4579-93ca-b64d2aaf4d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:14.902625 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932ef20-b1fb-4579-93ca-b64d2aaf4d8c admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:14.902903 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932ef20-b1fb-4579-93ca-b64d2aaf4d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:14.903174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932ef20-b1fb-4579-93ca-b64d2aaf4d8c admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:14.903882 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932ef20-b1fb-4579-93ca-b64d2aaf4d8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:14:14.904184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932ef20-b1fb-4579-93ca-b64d2aaf4d8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:14.905806 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932ef20-b1fb-4579-93ca-b64d2aaf4d8c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:14.917174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1432/2864] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:14:14.922066 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db680f-a2d5-4514-ab31-c2b1519b25c7 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:14.933440 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db680f-a2d5-4514-ab31-c2b1519b25c7 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:14.938827 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5af65-a68b-42eb-9e0e-3af6435dec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:14.939500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5af65-a68b-42eb-9e0e-3af6435dec75 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:14.939785 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5af65-a68b-42eb-9e0e-3af6435dec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:14.940064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5af65-a68b-42eb-9e0e-3af6435dec75 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:14.940798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5af65-a68b-42eb-9e0e-3af6435dec75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:14.942710 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5af65-a68b-42eb-9e0e-3af6435dec75 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:14.969846 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1433/2865] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:14:14 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:14:14.974031 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932ef20-b1fb-4579-93ca-b64d2aaf4d8c admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:14.985020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932ef20-b1fb-4579-93ca-b64d2aaf4d8c admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:15.040306 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f608d-1986-41b6-bdd0-601d0427db8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:15.040978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f608d-1986-41b6-bdd0-601d0427db8d admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:15.041202 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f608d-1986-41b6-bdd0-601d0427db8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:15.041478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f608d-1986-41b6-bdd0-601d0427db8d admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:15.042175 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f608d-1986-41b6-bdd0-601d0427db8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:15.045318 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f608d-1986-41b6-bdd0-601d0427db8d admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:15.065938 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1433/2866] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:14:14 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:14:15.070278 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5af65-a68b-42eb-9e0e-3af6435dec75 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:15.081027 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5af65-a68b-42eb-9e0e-3af6435dec75 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:15.085757 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617e5ac-fccc-4a69-8d0e-b0b1a302f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:15.086415 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617e5ac-fccc-4a69-8d0e-b0b1a302f28d admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:15.086705 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617e5ac-fccc-4a69-8d0e-b0b1a302f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:15.086985 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617e5ac-fccc-4a69-8d0e-b0b1a302f28d admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:15.087707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617e5ac-fccc-4a69-8d0e-b0b1a302f28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:15.089341 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617e5ac-fccc-4a69-8d0e-b0b1a302f28d admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:15.100497 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1434/2867] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:14:15.104813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f608d-1986-41b6-bdd0-601d0427db8d admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:15.115759 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f608d-1986-41b6-bdd0-601d0427db8d admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:15.120827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab2113-7916-4188-9780-c45c44cc4699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:15.121608 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab2113-7916-4188-9780-c45c44cc4699 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:15.121827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab2113-7916-4188-9780-c45c44cc4699 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:15.122174 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab2113-7916-4188-9780-c45c44cc4699 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:15.122944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab2113-7916-4188-9780-c45c44cc4699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:15.124585 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab2113-7916-4188-9780-c45c44cc4699 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:15.135312 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1434/2868] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:14:15.141054 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617e5ac-fccc-4a69-8d0e-b0b1a302f28d admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:15.151274 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617e5ac-fccc-4a69-8d0e-b0b1a302f28d admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:15.156662 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e92f-479e-465d-a773-7a1846ea28e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:15.157547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e92f-479e-465d-a773-7a1846ea28e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:15.157837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e92f-479e-465d-a773-7a1846ea28e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:15.158112 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83e92f-479e-465d-a773-7a1846ea28e1 admin admin] PATH_INFO: `/v3/projects/f6b741ab4e1143ab8ca64ab051094354/users/ab13e048111840e992700f3a335203cd/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:15.173849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83e92f-479e-465d-a773-7a1846ea28e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b741ab4e1143ab8ca64ab051094354, user_id=ab13e048111840e992700f3a335203cd, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:15.176531 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83e92f-479e-465d-a773-7a1846ea28e1 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:15.190403 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1435/2869] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:14:15 2022] PUT /identity/v3/projects/f6b741ab4e1143ab8ca64ab051094354/users/ab13e048111840e992700f3a335203cd/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:14:15.201182 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef50c3d-8f5c-4456-9225-41ca653931f3 None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:15.201527 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef50c3d-8f5c-4456-9225-41ca653931f3 None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:15.201813 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef50c3d-8f5c-4456-9225-41ca653931f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:15.231698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-cef50c3d-8f5c-4456-9225-41ca653931f3 None None] MFA Rules not processed for user `ab13e048111840e992700f3a335203cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:15.232977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-cef50c3d-8f5c-4456-9225-41ca653931f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:15.299056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1435/2870] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:15.310312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e92f-479e-465d-a773-7a1846ea28e1 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:15.312119 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ca83e92f-479e-465d-a773-7a1846ea28e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:15.338739 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:14:15.372781 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83e92f-479e-465d-a773-7a1846ea28e1 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:15.376535 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45e49c-55d6-4313-a27d-c16744b85992 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e35e7720e3bf4c9e80bdefa6a1cb8296', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:15.377219 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45e49c-55d6-4313-a27d-c16744b85992 service glance] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:15.377503 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45e49c-55d6-4313-a27d-c16744b85992 service glance] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:15.378106 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45e49c-55d6-4313-a27d-c16744b85992 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:15.387011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45e49c-55d6-4313-a27d-c16744b85992 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:15.388477 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45e49c-55d6-4313-a27d-c16744b85992 service glance] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:15.397513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1436/2871] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:14:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:14:25.295475 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:14:25.393461 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=17 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:14:48.416465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef50c3d-8f5c-4456-9225-41ca653931f3 None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:48.478906 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef50c3d-8f5c-4456-9225-41ca653931f3 None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:48.484474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:48.485342 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:48.485627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:48.485905 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] PATH_INFO: `/v3/users/ab13e048111840e992700f3a335203cd` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:48.487098 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab13e048111840e992700f3a335203cd)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:48.489036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:48.536465 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab13e048111840e992700f3a335203cd'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:48.545248 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab13e048111840e992700f3a335203cd'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:48.552233 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab13e048111840e992700f3a335203cd'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:48.552910 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab13e048111840e992700f3a335203cd'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:48.565374 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab13e048111840e992700f3a335203cd'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:48.571298 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1436/2872] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:14:48 2022] DELETE /identity/v3/users/ab13e048111840e992700f3a335203cd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:14:48.576206 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d4a75-857d-45b3-a9ed-603dc6b9abd8 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:48.576742 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d4a75-857d-45b3-a9ed-603dc6b9abd8 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:48.576864 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d4a75-857d-45b3-a9ed-603dc6b9abd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:48.595166 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-582d4a75-857d-45b3-a9ed-603dc6b9abd8 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:48.596430 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-582d4a75-857d-45b3-a9ed-603dc6b9abd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:48.662514 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1437/2873] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:48.674793 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:48.676474 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:48.736426 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2de307-31ca-4977-a06b-9c43eb851321 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:48.739996 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8f83d-7eb7-4905-864b-957a1202c637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:48.740677 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8f83d-7eb7-4905-864b-957a1202c637 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:48.740961 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8f83d-7eb7-4905-864b-957a1202c637 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:48.741239 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8f83d-7eb7-4905-864b-957a1202c637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:48.750741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8f83d-7eb7-4905-864b-957a1202c637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:48.752171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8f83d-7eb7-4905-864b-957a1202c637 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:48.761985 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1437/2874] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:14:49.037141 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d4a75-857d-45b3-a9ed-603dc6b9abd8 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:49.038564 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-582d4a75-857d-45b3-a9ed-603dc6b9abd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:49.050283 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d4a75-857d-45b3-a9ed-603dc6b9abd8 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:49.055052 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:49.055904 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.056228 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.057368 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] PATH_INFO: `/v3/projects/f6b741ab4e1143ab8ca64ab051094354` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.058798 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b741ab4e1143ab8ca64ab051094354)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:49.061240 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:49.112280 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b741ab4e1143ab8ca64ab051094354', 'request_id': 'req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:49.120145 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b741ab4e1143ab8ca64ab051094354', 'request_id': 'req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:49.128408 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b741ab4e1143ab8ca64ab051094354', 'request_id': 'req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:49.139915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] The token cache is being invalidate because project f6b741ab4e1143ab8ca64ab051094354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:14:49.140211 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:49.140905 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-9db0ded8-da22-4d6d-8ef3-641af53cfa8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:49.142479 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1438/2875] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:14:49 2022] DELETE /identity/v3/projects/f6b741ab4e1143ab8ca64ab051094354 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:14:49.147531 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd1844-d611-411b-b02e-15a1866a285d None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.147840 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd1844-d611-411b-b02e-15a1866a285d None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.148116 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd1844-d611-411b-b02e-15a1866a285d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.167349 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-83dd1844-d611-411b-b02e-15a1866a285d None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:49.168599 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-83dd1844-d611-411b-b02e-15a1866a285d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:49.190532 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:14:49.236023 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1438/2876] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:49.244195 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de774076-ff27-4d44-bf03-9c19a4a0eb51 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.244524 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de774076-ff27-4d44-bf03-9c19a4a0eb51 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.244815 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de774076-ff27-4d44-bf03-9c19a4a0eb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.264213 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-de774076-ff27-4d44-bf03-9c19a4a0eb51 None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:49.265506 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-de774076-ff27-4d44-bf03-9c19a4a0eb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:49.273769 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1439/2877] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:49.277843 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd1844-d611-411b-b02e-15a1866a285d None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:49.288032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd1844-d611-411b-b02e-15a1866a285d None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:49.292796 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803882a-38b1-4d4d-a9c6-0a0136ac2d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:49.293477 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803882a-38b1-4d4d-a9c6-0a0136ac2d6e admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.294292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803882a-38b1-4d4d-a9c6-0a0136ac2d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.294573 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803882a-38b1-4d4d-a9c6-0a0136ac2d6e admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.295293 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803882a-38b1-4d4d-a9c6-0a0136ac2d6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:14:49.295641 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803882a-38b1-4d4d-a9c6-0a0136ac2d6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:49.297309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803882a-38b1-4d4d-a9c6-0a0136ac2d6e admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:49.309123 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1439/2878] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:14:49.314150 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de774076-ff27-4d44-bf03-9c19a4a0eb51 None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:49.323547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de774076-ff27-4d44-bf03-9c19a4a0eb51 None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:49.329220 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea570e4f-e5d8-4bdc-9bfe-f27d5e6452c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:49.329870 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea570e4f-e5d8-4bdc-9bfe-f27d5e6452c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.330158 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea570e4f-e5d8-4bdc-9bfe-f27d5e6452c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.330441 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea570e4f-e5d8-4bdc-9bfe-f27d5e6452c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.331134 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea570e4f-e5d8-4bdc-9bfe-f27d5e6452c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:49.333072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea570e4f-e5d8-4bdc-9bfe-f27d5e6452c8 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:49.360140 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1440/2879] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:14:49 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:14:49.364043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803882a-38b1-4d4d-a9c6-0a0136ac2d6e admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:49.374987 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803882a-38b1-4d4d-a9c6-0a0136ac2d6e admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:49.432188 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a9f5f-ff2c-4569-9c18-eaf9209c1d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:49.432932 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a9f5f-ff2c-4569-9c18-eaf9209c1d35 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.433165 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a9f5f-ff2c-4569-9c18-eaf9209c1d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.433456 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a9f5f-ff2c-4569-9c18-eaf9209c1d35 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.434144 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a9f5f-ff2c-4569-9c18-eaf9209c1d35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:49.436752 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a9f5f-ff2c-4569-9c18-eaf9209c1d35 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:49.457826 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1440/2880] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:14:49 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:14:49.462025 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea570e4f-e5d8-4bdc-9bfe-f27d5e6452c8 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:49.472312 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea570e4f-e5d8-4bdc-9bfe-f27d5e6452c8 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:49.477001 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1ca35-b053-403a-873d-cf91005b6ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:49.477654 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1ca35-b053-403a-873d-cf91005b6ecf admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.477940 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1ca35-b053-403a-873d-cf91005b6ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.478243 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1ca35-b053-403a-873d-cf91005b6ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.479058 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1ca35-b053-403a-873d-cf91005b6ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:49.480695 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1ca35-b053-403a-873d-cf91005b6ecf admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:49.492491 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1441/2881] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:14:49.497462 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a9f5f-ff2c-4569-9c18-eaf9209c1d35 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:49.507814 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a9f5f-ff2c-4569-9c18-eaf9209c1d35 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:49.512811 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036cc0a-f1d0-4a84-86a5-b4f2edb794bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:49.513537 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036cc0a-f1d0-4a84-86a5-b4f2edb794bc admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.513825 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036cc0a-f1d0-4a84-86a5-b4f2edb794bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.514105 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036cc0a-f1d0-4a84-86a5-b4f2edb794bc admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.514839 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036cc0a-f1d0-4a84-86a5-b4f2edb794bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:49.516476 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036cc0a-f1d0-4a84-86a5-b4f2edb794bc admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:49.526557 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1441/2882] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:14:49.531536 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1ca35-b053-403a-873d-cf91005b6ecf admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:49.542010 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1ca35-b053-403a-873d-cf91005b6ecf admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:49.547353 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35956b4e-cb5a-42fe-a971-6f4230ddf660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:49.548226 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956b4e-cb5a-42fe-a971-6f4230ddf660 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.548547 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956b4e-cb5a-42fe-a971-6f4230ddf660 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.548830 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956b4e-cb5a-42fe-a971-6f4230ddf660 admin admin] PATH_INFO: `/v3/projects/80c969408535439ab92c86bfef3e5226/users/c9ed029dbb65471ca9ab340a1f3bb5e0/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.564541 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35956b4e-cb5a-42fe-a971-6f4230ddf660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c969408535439ab92c86bfef3e5226, user_id=c9ed029dbb65471ca9ab340a1f3bb5e0, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:49.567187 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35956b4e-cb5a-42fe-a971-6f4230ddf660 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:49.581076 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1442/2883] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:14:49 2022] PUT /identity/v3/projects/80c969408535439ab92c86bfef3e5226/users/c9ed029dbb65471ca9ab340a1f3bb5e0/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:14:49.591955 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a213df-392a-43eb-9ab0-5bb44649d9ce None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.592258 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a213df-392a-43eb-9ab0-5bb44649d9ce None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.592564 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a213df-392a-43eb-9ab0-5bb44649d9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.619896 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-51a213df-392a-43eb-9ab0-5bb44649d9ce None None] MFA Rules not processed for user `c9ed029dbb65471ca9ab340a1f3bb5e0`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:49.621149 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-51a213df-392a-43eb-9ab0-5bb44649d9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:49.685236 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1442/2884] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 21:14:49.696615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35956b4e-cb5a-42fe-a971-6f4230ddf660 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:49.698319 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-35956b4e-cb5a-42fe-a971-6f4230ddf660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:49.721810 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:14:49.758572 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35956b4e-cb5a-42fe-a971-6f4230ddf660 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:49.762457 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0de3a-1d15-4fae-a2c2-26b01b9c92ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:49.763098 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0de3a-1d15-4fae-a2c2-26b01b9c92ec service cinder] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.763379 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0de3a-1d15-4fae-a2c2-26b01b9c92ec service cinder] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.763656 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0de3a-1d15-4fae-a2c2-26b01b9c92ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.771875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0de3a-1d15-4fae-a2c2-26b01b9c92ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:49.773310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0de3a-1d15-4fae-a2c2-26b01b9c92ec service cinder] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:49.782694 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1443/2885] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:14:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:14:49.807540 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a213df-392a-43eb-9ab0-5bb44649d9ce None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:49.826417 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:14:49.873517 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a213df-392a-43eb-9ab0-5bb44649d9ce None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:49.879205 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:49.880095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.880407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.880710 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] PATH_INFO: `/v3/users/c9ed029dbb65471ca9ab340a1f3bb5e0` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.881946 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9ed029dbb65471ca9ab340a1f3bb5e0)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:49.883857 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:49.930176 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9ed029dbb65471ca9ab340a1f3bb5e0'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:49.937926 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9ed029dbb65471ca9ab340a1f3bb5e0'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:49.945231 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9ed029dbb65471ca9ab340a1f3bb5e0'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:49.945877 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9ed029dbb65471ca9ab340a1f3bb5e0'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:49.958890 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9ed029dbb65471ca9ab340a1f3bb5e0'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:49.964964 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1443/2886] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:14:49 2022] DELETE /identity/v3/users/c9ed029dbb65471ca9ab340a1f3bb5e0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:14:49.970478 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef245054-f187-4c10-a95c-d8afae7b01bc None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:49.970811 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef245054-f187-4c10-a95c-d8afae7b01bc None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:49.971038 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef245054-f187-4c10-a95c-d8afae7b01bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:49.988273 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-ef245054-f187-4c10-a95c-d8afae7b01bc None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:49.989501 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ef245054-f187-4c10-a95c-d8afae7b01bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:50.049104 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1444/2887] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:50.059084 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:50.060604 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:50.118751 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820eea0-c4c5-455c-86ab-f32f78377df7 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:50.122522 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddf801-8b1e-487c-9aed-0a2df1cff868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:50.123167 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddf801-8b1e-487c-9aed-0a2df1cff868 service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:50.123454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddf801-8b1e-487c-9aed-0a2df1cff868 service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:50.123733 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddf801-8b1e-487c-9aed-0a2df1cff868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:50.133263 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddf801-8b1e-487c-9aed-0a2df1cff868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:50.134787 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddf801-8b1e-487c-9aed-0a2df1cff868 service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:50.145485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1444/2888] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:14:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:14:50.415760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef245054-f187-4c10-a95c-d8afae7b01bc None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:50.417364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-ef245054-f187-4c10-a95c-d8afae7b01bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:50.431841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef245054-f187-4c10-a95c-d8afae7b01bc None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:50.436776 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:50.438207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:50.438505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:50.438792 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] PATH_INFO: `/v3/projects/80c969408535439ab92c86bfef3e5226` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:50.440300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80c969408535439ab92c86bfef3e5226)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:50.442275 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:50.490669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80c969408535439ab92c86bfef3e5226', 'request_id': 'req-fd983951-f835-458e-a61c-ab4df6064dd7'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:50.498414 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80c969408535439ab92c86bfef3e5226', 'request_id': 'req-fd983951-f835-458e-a61c-ab4df6064dd7'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:50.505008 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80c969408535439ab92c86bfef3e5226', 'request_id': 'req-fd983951-f835-458e-a61c-ab4df6064dd7'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:50.516014 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] The token cache is being invalidate because project 80c969408535439ab92c86bfef3e5226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:14:50.516313 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:50.517027 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-fd983951-f835-458e-a61c-ab4df6064dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:14:50.518615 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1445/2889] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:14:50 2022] DELETE /identity/v3/projects/80c969408535439ab92c86bfef3e5226 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:14:50.525304 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd981f-dde8-4199-8b9c-58538be5bc8b None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:50.525921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd981f-dde8-4199-8b9c-58538be5bc8b None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:50.525921 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd981f-dde8-4199-8b9c-58538be5bc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:50.544918 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-a8fd981f-dde8-4199-8b9c-58538be5bc8b None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:50.546118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-a8fd981f-dde8-4199-8b9c-58538be5bc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:50.578639 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:14:50.606904 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1445/2890] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:50.616526 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed116c-18f5-428f-9ab5-1d3bcaa2c1ee None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:50.617118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed116c-18f5-428f-9ab5-1d3bcaa2c1ee None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:50.617118 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed116c-18f5-428f-9ab5-1d3bcaa2c1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:50.636746 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-d2ed116c-18f5-428f-9ab5-1d3bcaa2c1ee None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:50.637983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-d2ed116c-18f5-428f-9ab5-1d3bcaa2c1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:50.646394 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1446/2891] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:50.651192 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd981f-dde8-4199-8b9c-58538be5bc8b None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:50.662096 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd981f-dde8-4199-8b9c-58538be5bc8b None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:50.667082 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8d087-85b4-4eb5-9d20-d01e3da50332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:50.667741 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8d087-85b4-4eb5-9d20-d01e3da50332 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:50.668044 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8d087-85b4-4eb5-9d20-d01e3da50332 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:50.668308 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8d087-85b4-4eb5-9d20-d01e3da50332 admin admin] PATH_INFO: `/v3/domains` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:50.669693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8d087-85b4-4eb5-9d20-d01e3da50332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86669) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 21:14:50.670014 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8d087-85b4-4eb5-9d20-d01e3da50332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:50.671627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8d087-85b4-4eb5-9d20-d01e3da50332 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:50.685653 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1446/2892] 10.222.0.85 () {58 vars in 1152 bytes} [Tue Jul 19 21:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 21:14:50.690651 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed116c-18f5-428f-9ab5-1d3bcaa2c1ee None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:50.701568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed116c-18f5-428f-9ab5-1d3bcaa2c1ee None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:50.706277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979addcd-3099-4c67-aff2-bae3a8f7e9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:50.706926 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979addcd-3099-4c67-aff2-bae3a8f7e9e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:50.707207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979addcd-3099-4c67-aff2-bae3a8f7e9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:50.707483 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979addcd-3099-4c67-aff2-bae3a8f7e9e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:50.708181 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979addcd-3099-4c67-aff2-bae3a8f7e9e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:50.710094 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979addcd-3099-4c67-aff2-bae3a8f7e9e0 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:50.736936 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1447/2893] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:14:50 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 21:14:50.740892 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8d087-85b4-4eb5-9d20-d01e3da50332 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:50.751111 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8d087-85b4-4eb5-9d20-d01e3da50332 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:50.807010 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e04547-7ad3-4ace-ba06-9a610a678342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:50.807675 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e04547-7ad3-4ace-ba06-9a610a678342 admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:50.807956 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e04547-7ad3-4ace-ba06-9a610a678342 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:50.808230 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e04547-7ad3-4ace-ba06-9a610a678342 admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:50.808944 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e04547-7ad3-4ace-ba06-9a610a678342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:50.810851 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e04547-7ad3-4ace-ba06-9a610a678342 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:50.831288 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1447/2894] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:14:50 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:14:50.835417 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979addcd-3099-4c67-aff2-bae3a8f7e9e0 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:50.846908 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979addcd-3099-4c67-aff2-bae3a8f7e9e0 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:50.851607 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c48dd-3fa4-412d-8a9a-08cf16f182bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:50.852265 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c48dd-3fa4-412d-8a9a-08cf16f182bf admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:50.852588 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c48dd-3fa4-412d-8a9a-08cf16f182bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:50.852871 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c48dd-3fa4-412d-8a9a-08cf16f182bf admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:50.853600 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c48dd-3fa4-412d-8a9a-08cf16f182bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:50.855202 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c48dd-3fa4-412d-8a9a-08cf16f182bf admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:50.866384 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1448/2895] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:14:50.870406 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e04547-7ad3-4ace-ba06-9a610a678342 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:50.881282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e04547-7ad3-4ace-ba06-9a610a678342 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:50.886032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5b213-525d-4a25-b12a-6b83d4d2d2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:50.886693 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5b213-525d-4a25-b12a-6b83d4d2d2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:50.886972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5b213-525d-4a25-b12a-6b83d4d2d2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:50.887255 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5b213-525d-4a25-b12a-6b83d4d2d2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:50.888004 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5b213-525d-4a25-b12a-6b83d4d2d2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:50.889645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5b213-525d-4a25-b12a-6b83d4d2d2b3 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:50.899603 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1448/2896] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:14:50.903968 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c48dd-3fa4-412d-8a9a-08cf16f182bf admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:50.914729 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c48dd-3fa4-412d-8a9a-08cf16f182bf admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:50.920460 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0a22d-0192-4d05-bbb7-1abf4f85f1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:50.921413 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a22d-0192-4d05-bbb7-1abf4f85f1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:50.921724 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a22d-0192-4d05-bbb7-1abf4f85f1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:50.922033 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a22d-0192-4d05-bbb7-1abf4f85f1e6 admin admin] PATH_INFO: `/v3/projects/8b9d58fb2dee4edf9632db6ee83312fb/users/1c61ac31c3a24147ac629bd19f9cb9dc/roles/0f8e73b285204863a74f8af4ff7bbc86` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:50.924433 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:14:50.940295 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0a22d-0192-4d05-bbb7-1abf4f85f1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9d58fb2dee4edf9632db6ee83312fb, user_id=1c61ac31c3a24147ac629bd19f9cb9dc, role_id=0f8e73b285204863a74f8af4ff7bbc86)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:50.942993 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0a22d-0192-4d05-bbb7-1abf4f85f1e6 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:50.957424 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1449/2897] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:14:50 2022] PUT /identity/v3/projects/8b9d58fb2dee4edf9632db6ee83312fb/users/1c61ac31c3a24147ac629bd19f9cb9dc/roles/0f8e73b285204863a74f8af4ff7bbc86 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:14:50.962586 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3031be7-b593-4c6c-aa37-976056a60dce None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:50.962881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3031be7-b593-4c6c-aa37-976056a60dce None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:50.963195 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3031be7-b593-4c6c-aa37-976056a60dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:50.982421 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-e3031be7-b593-4c6c-aa37-976056a60dce None None] MFA Rules not processed for user `adee0d7d3c614774bcfa96ea7aed77e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:50.983636 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e3031be7-b593-4c6c-aa37-976056a60dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:51.053530 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1449/2898] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:51.064018 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0a22d-0192-4d05-bbb7-1abf4f85f1e6 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:51.065915 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-c0b0a22d-0192-4d05-bbb7-1abf4f85f1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:51.126499 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0a22d-0192-4d05-bbb7-1abf4f85f1e6 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:51.130170 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157b973-c033-49f6-930d-f2e618707d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:51.130820 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157b973-c033-49f6-930d-f2e618707d09 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:51.131109 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157b973-c033-49f6-930d-f2e618707d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:51.131389 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157b973-c033-49f6-930d-f2e618707d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:51.141768 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157b973-c033-49f6-930d-f2e618707d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:51.143198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157b973-c033-49f6-930d-f2e618707d09 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:51.154285 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1450/2899] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:14:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:14:55.179629 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79094df-4dc5-4a21-8b2f-7e3946abbbaa None None] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:55.180183 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79094df-4dc5-4a21-8b2f-7e3946abbbaa None None] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:55.180292 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79094df-4dc5-4a21-8b2f-7e3946abbbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:55.214185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.auth.core [None req-c79094df-4dc5-4a21-8b2f-7e3946abbbaa None None] MFA Rules not processed for user `1c61ac31c3a24147ac629bd19f9cb9dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86669) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:14:55.214185 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-c79094df-4dc5-4a21-8b2f-7e3946abbbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:14:55.250437 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:14:55.286977 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1450/2900] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:14:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:14:55.291504 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157b973-c033-49f6-930d-f2e618707d09 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:55.301816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157b973-c033-49f6-930d-f2e618707d09 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:55.306543 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f38f7d-c4f9-41e6-98d1-b5b691ed14c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:55.307207 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f38f7d-c4f9-41e6-98d1-b5b691ed14c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:55.307497 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f38f7d-c4f9-41e6-98d1-b5b691ed14c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:55.307780 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f38f7d-c4f9-41e6-98d1-b5b691ed14c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:55.308709 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f38f7d-c4f9-41e6-98d1-b5b691ed14c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:55.310632 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f38f7d-c4f9-41e6-98d1-b5b691ed14c2 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:55.338558 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1451/2901] 10.222.0.85 () {60 vars in 1152 bytes} [Tue Jul 19 21:14:55 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:14:55.343457 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79094df-4dc5-4a21-8b2f-7e3946abbbaa None None] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:55.356684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79094df-4dc5-4a21-8b2f-7e3946abbbaa None None] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:55.420952 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac098efc-55bd-4b3d-8710-046ddac9248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:55.421538 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac098efc-55bd-4b3d-8710-046ddac9248f admin admin] REQUEST_METHOD: `POST` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:55.421824 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac098efc-55bd-4b3d-8710-046ddac9248f admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:55.422128 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac098efc-55bd-4b3d-8710-046ddac9248f admin admin] PATH_INFO: `/v3/users` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:55.422822 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac098efc-55bd-4b3d-8710-046ddac9248f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:55.424775 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac098efc-55bd-4b3d-8710-046ddac9248f admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:55.444818 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1451/2902] 10.222.0.85 () {60 vars in 1143 bytes} [Tue Jul 19 21:14:55 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 21:14:55.449244 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f38f7d-c4f9-41e6-98d1-b5b691ed14c2 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:55.460315 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f38f7d-c4f9-41e6-98d1-b5b691ed14c2 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:55.466467 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a118d03-7716-4ade-9ad5-2b47f4f98db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:55.467381 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a118d03-7716-4ade-9ad5-2b47f4f98db1 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:55.467669 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a118d03-7716-4ade-9ad5-2b47f4f98db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:55.467953 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a118d03-7716-4ade-9ad5-2b47f4f98db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:55.468761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a118d03-7716-4ade-9ad5-2b47f4f98db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:55.470358 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a118d03-7716-4ade-9ad5-2b47f4f98db1 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:55.481918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1452/2903] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 21:14:55.486384 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac098efc-55bd-4b3d-8710-046ddac9248f admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:55.496909 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac098efc-55bd-4b3d-8710-046ddac9248f admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:55.501666 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402dc3b-db42-4328-8cde-fff1d2d0a06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:55.502549 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402dc3b-db42-4328-8cde-fff1d2d0a06c admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:55.502829 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402dc3b-db42-4328-8cde-fff1d2d0a06c admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:55.503104 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402dc3b-db42-4328-8cde-fff1d2d0a06c admin admin] PATH_INFO: `/v3/projects/a4529614c5c64e9eae1ae8e02570e4f9/users/e38289ca8c7349f09b246b13d290bf34/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:55.518838 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402dc3b-db42-4328-8cde-fff1d2d0a06c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4529614c5c64e9eae1ae8e02570e4f9, user_id=e38289ca8c7349f09b246b13d290bf34, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:55.521488 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402dc3b-db42-4328-8cde-fff1d2d0a06c admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:55.535642 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1452/2904] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:14:55 2022] PUT /identity/v3/projects/a4529614c5c64e9eae1ae8e02570e4f9/users/e38289ca8c7349f09b246b13d290bf34/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 21:14:55.539715 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a118d03-7716-4ade-9ad5-2b47f4f98db1 admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:55.550196 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a118d03-7716-4ade-9ad5-2b47f4f98db1 admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:55.607702 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb73e7-457b-47fb-9893-d52daa6f5994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:55.608401 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb73e7-457b-47fb-9893-d52daa6f5994 admin admin] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:55.608687 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb73e7-457b-47fb-9893-d52daa6f5994 admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:55.608967 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb73e7-457b-47fb-9893-d52daa6f5994 admin admin] PATH_INFO: `/v3/roles` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:55.609701 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb73e7-457b-47fb-9893-d52daa6f5994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:55.611286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb73e7-457b-47fb-9893-d52daa6f5994 admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:55.621642 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1453/2905] 10.222.0.85 () {58 vars in 1121 bytes} [Tue Jul 19 21:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 21:14:55.626338 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402dc3b-db42-4328-8cde-fff1d2d0a06c admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:14:55.637036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402dc3b-db42-4328-8cde-fff1d2d0a06c admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:14:55.642538 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16ab5-ac03-4435-8f67-7cb175b21bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:14:55.643412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16ab5-ac03-4435-8f67-7cb175b21bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:14:55.643694 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16ab5-ac03-4435-8f67-7cb175b21bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:14:55.643972 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16ab5-ac03-4435-8f67-7cb175b21bb7 admin admin] PATH_INFO: `/v3/projects/a4529614c5c64e9eae1ae8e02570e4f9/users/e38289ca8c7349f09b246b13d290bf34/roles/441f27d47b2f45e3a60f724e615a42f7` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:14:55.646032 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c16ab5-ac03-4435-8f67-7cb175b21bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4529614c5c64e9eae1ae8e02570e4f9, user_id=e38289ca8c7349f09b246b13d290bf34, role_id=441f27d47b2f45e3a60f724e615a42f7)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:14:55.648698 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c16ab5-ac03-4435-8f67-7cb175b21bb7 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:14:55.659405 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1453/2906] 10.222.0.85 () {60 vars in 1482 bytes} [Tue Jul 19 21:14:55 2022] PUT /identity/v3/projects/a4529614c5c64e9eae1ae8e02570e4f9/users/e38289ca8c7349f09b246b13d290bf34/roles/441f27d47b2f45e3a60f724e615a42f7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:15:00.560449 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c7292-c5fc-47ae-a41d-f8a569ff3bc1 None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:15:00.560983 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c7292-c5fc-47ae-a41d-f8a569ff3bc1 None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:15:00.561880 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c7292-c5fc-47ae-a41d-f8a569ff3bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:15:00.605902 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-286c7292-c5fc-47ae-a41d-f8a569ff3bc1 None None] MFA Rules not processed for user `e38289ca8c7349f09b246b13d290bf34`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:15:00.607631 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-286c7292-c5fc-47ae-a41d-f8a569ff3bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:00.660185 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:15:00.704916 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1454/2907] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:15:00.725228 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16ab5-ac03-4435-8f67-7cb175b21bb7 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:15:00.729196 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-99c16ab5-ac03-4435-8f67-7cb175b21bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:00.815628 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16ab5-ac03-4435-8f67-7cb175b21bb7 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:15:00.822107 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef615d-39b5-4a2a-aa45-460fee269627 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:15:00.822770 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef615d-39b5-4a2a-aa45-460fee269627 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:15:00.823055 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef615d-39b5-4a2a-aa45-460fee269627 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:15:00.823341 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef615d-39b5-4a2a-aa45-460fee269627 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:15:00.851407 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:15:00.890564 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef615d-39b5-4a2a-aa45-460fee269627 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:15:00.892201 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef615d-39b5-4a2a-aa45-460fee269627 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:15:00.903579 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1454/2908] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:15:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:15:01.935758 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d98c92-0457-4fa0-aef9-0cebbbf69fbf None None] REQUEST_METHOD: `POST` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:15:01.936752 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d98c92-0457-4fa0-aef9-0cebbbf69fbf None None] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:15:01.937147 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d98c92-0457-4fa0-aef9-0cebbbf69fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:15:01.957131 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.auth.core [None req-18d98c92-0457-4fa0-aef9-0cebbbf69fbf None None] MFA Rules not processed for user `1c61ac31c3a24147ac629bd19f9cb9dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86668) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 21:15:01.958465 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-18d98c92-0457-4fa0-aef9-0cebbbf69fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:01.967147 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1455/2909] 10.222.0.85 () {58 vars in 957 bytes} [Tue Jul 19 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 21:15:10.898276 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=16 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:15:11.951626 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=15 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:15:23.528008 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef615d-39b5-4a2a-aa45-460fee269627 service nova] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:15:23.529795 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-d9ef615d-39b5-4a2a-aa45-460fee269627 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:23.593466 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef615d-39b5-4a2a-aa45-460fee269627 service nova] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:15:23.597282 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d0052-6601-467b-9e90-090d14e20133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81411204bc9f4210880d807b41da9643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:15:23.597937 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d0052-6601-467b-9e90-090d14e20133 service cinder] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:15:23.598220 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d0052-6601-467b-9e90-090d14e20133 service cinder] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:15:23.598570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d0052-6601-467b-9e90-090d14e20133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:15:23.600762 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e33d0052-6601-467b-9e90-090d14e20133 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:23.617208 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d0052-6601-467b-9e90-090d14e20133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:15:23.618645 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d0052-6601-467b-9e90-090d14e20133 service cinder] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:15:23.633097 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1455/2910] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:15:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:15:23.956707 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d98c92-0457-4fa0-aef9-0cebbbf69fbf None None] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:15:23.958286 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-18d98c92-0457-4fa0-aef9-0cebbbf69fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:24.025619 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d98c92-0457-4fa0-aef9-0cebbbf69fbf None None] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:15:24.029991 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0bb9b-3fbe-4ec3-aa41-d6b6a6a7e650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:15:24.030644 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0bb9b-3fbe-4ec3-aa41-d6b6a6a7e650 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:15:24.030922 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0bb9b-3fbe-4ec3-aa41-d6b6a6a7e650 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:15:24.031200 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0bb9b-3fbe-4ec3-aa41-d6b6a6a7e650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:15:24.033085 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-02c0bb9b-3fbe-4ec3-aa41-d6b6a6a7e650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:24.044997 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0bb9b-3fbe-4ec3-aa41-d6b6a6a7e650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:15:24.046447 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0bb9b-3fbe-4ec3-aa41-d6b6a6a7e650 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:15:24.056537 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1456/2911] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:15:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:15:24.627754 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d0052-6601-467b-9e90-090d14e20133 service cinder] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:15:24.631146 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-e33d0052-6601-467b-9e90-090d14e20133 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:24.638842 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0bb9b-3fbe-4ec3-aa41-d6b6a6a7e650 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:15:24.653035 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0bb9b-3fbe-4ec3-aa41-d6b6a6a7e650 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:15:24.664883 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54ca96-ded7-410d-b71c-bf5b296c48b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:15:24.665998 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54ca96-ded7-410d-b71c-bf5b296c48b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:15:24.666505 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54ca96-ded7-410d-b71c-bf5b296c48b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:15:24.667016 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54ca96-ded7-410d-b71c-bf5b296c48b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:15:24.670306 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-3c54ca96-ded7-410d-b71c-bf5b296c48b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:24.695731 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54ca96-ded7-410d-b71c-bf5b296c48b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:15:24.698516 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54ca96-ded7-410d-b71c-bf5b296c48b0 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:15:24.719939 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1457/2912] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:15:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:15:24.750900 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d0052-6601-467b-9e90-090d14e20133 service cinder] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:15:24.760962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c0b36-2145-4f7a-951d-3932944d4235 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:15:24.762711 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c0b36-2145-4f7a-951d-3932944d4235 service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:15:24.763234 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c0b36-2145-4f7a-951d-3932944d4235 service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:15:24.763742 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c0b36-2145-4f7a-951d-3932944d4235 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:15:24.767319 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-688c0b36-2145-4f7a-951d-3932944d4235 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:24.790827 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c0b36-2145-4f7a-951d-3932944d4235 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:15:24.792734 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c0b36-2145-4f7a-951d-3932944d4235 service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:15:24.814519 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1456/2913] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:15:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:15:34.711127 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=32 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:15:34.804190 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=58 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:15:41.451364 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54ca96-ded7-410d-b71c-bf5b296c48b0 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:15:41.464108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54ca96-ded7-410d-b71c-bf5b296c48b0 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:15:41.470347 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc65c4d-663c-4a48-90d0-1c8f33a0b9d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:15:41.470995 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc65c4d-663c-4a48-90d0-1c8f33a0b9d7 service nova] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:15:41.471277 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc65c4d-663c-4a48-90d0-1c8f33a0b9d7 service nova] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:15:41.471553 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc65c4d-663c-4a48-90d0-1c8f33a0b9d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:15:41.473422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-8cc65c4d-663c-4a48-90d0-1c8f33a0b9d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:41.488672 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc65c4d-663c-4a48-90d0-1c8f33a0b9d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:15:41.490484 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc65c4d-663c-4a48-90d0-1c8f33a0b9d7 service nova] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:15:41.502300 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1458/2914] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:15:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:15:41.758279 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c0b36-2145-4f7a-951d-3932944d4235 service placement] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:15:41.771699 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c0b36-2145-4f7a-951d-3932944d4235 service placement] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:15:41.775469 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469edf-7ecb-4bb9-af39-2726f1d7411d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:15:41.776112 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469edf-7ecb-4bb9-af39-2726f1d7411d service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:15:41.776412 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469edf-7ecb-4bb9-af39-2726f1d7411d service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:15:41.776686 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469edf-7ecb-4bb9-af39-2726f1d7411d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:15:41.779141 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3b469edf-7ecb-4bb9-af39-2726f1d7411d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:41.790925 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b469edf-7ecb-4bb9-af39-2726f1d7411d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:15:41.792385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b469edf-7ecb-4bb9-af39-2726f1d7411d service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:15:41.801385 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1457/2915] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:15:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:15:45.036034 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc65c4d-663c-4a48-90d0-1c8f33a0b9d7 service nova] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:15:45.046055 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc65c4d-663c-4a48-90d0-1c8f33a0b9d7 service nova] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:15:45.049914 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de39b2e-3ea8-4df7-83cc-d2e269fb1d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:15:45.050560 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de39b2e-3ea8-4df7-83cc-d2e269fb1d3b service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:15:45.050841 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de39b2e-3ea8-4df7-83cc-d2e269fb1d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:15:45.051115 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de39b2e-3ea8-4df7-83cc-d2e269fb1d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:15:45.062454 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de39b2e-3ea8-4df7-83cc-d2e269fb1d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:15:45.063875 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de39b2e-3ea8-4df7-83cc-d2e269fb1d3b service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:15:45.073761 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1459/2916] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:15:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:15:46.960354 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469edf-7ecb-4bb9-af39-2726f1d7411d service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:15:46.962120 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3b469edf-7ecb-4bb9-af39-2726f1d7411d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:46.980744 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469edf-7ecb-4bb9-af39-2726f1d7411d service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:15:46.995570 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a7d2f-357d-48aa-a68b-4459435357ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:15:46.997056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a7d2f-357d-48aa-a68b-4459435357ac service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:15:46.997345 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a7d2f-357d-48aa-a68b-4459435357ac service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:15:46.997627 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a7d2f-357d-48aa-a68b-4459435357ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:15:46.999580 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-311a7d2f-357d-48aa-a68b-4459435357ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:15:47.013076 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a7d2f-357d-48aa-a68b-4459435357ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:15:47.014695 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a7d2f-357d-48aa-a68b-4459435357ac service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:15:47.026270 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1458/2917] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:15:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:15:55.069310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=12 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:15:57.021454 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=12 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:18:16.349393 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de39b2e-3ea8-4df7-83cc-d2e269fb1d3b service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:18:16.358978 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de39b2e-3ea8-4df7-83cc-d2e269fb1d3b service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:18:16.362693 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c0d7d-b150-4865-a778-f4c32ec5eab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:18:16.363344 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0d7d-b150-4865-a778-f4c32ec5eab1 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:18:16.363629 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0d7d-b150-4865-a778-f4c32ec5eab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:18:16.363906 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0d7d-b150-4865-a778-f4c32ec5eab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:18:16.365822 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-146c0d7d-b150-4865-a778-f4c32ec5eab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:18:16.375898 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c0d7d-b150-4865-a778-f4c32ec5eab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:18:16.377310 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c0d7d-b150-4865-a778-f4c32ec5eab1 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:18:16.387944 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1460/2918] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:18:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:18:16.491493 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a7d2f-357d-48aa-a68b-4459435357ac service nova] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:18:16.501596 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a7d2f-357d-48aa-a68b-4459435357ac service nova] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:18:16.505810 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67648b19-52d3-46c8-98f7-3728224da7ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:18:16.506478 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67648b19-52d3-46c8-98f7-3728224da7ce service neutron] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:18:16.506760 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67648b19-52d3-46c8-98f7-3728224da7ce service neutron] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:18:16.507038 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67648b19-52d3-46c8-98f7-3728224da7ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:18:16.509002 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-67648b19-52d3-46c8-98f7-3728224da7ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:18:16.520009 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67648b19-52d3-46c8-98f7-3728224da7ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:18:16.521508 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67648b19-52d3-46c8-98f7-3728224da7ce service neutron] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:18:16.531184 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1459/2919] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:18:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:18:16.649198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c0d7d-b150-4865-a778-f4c32ec5eab1 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:18:16.665954 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c0d7d-b150-4865-a778-f4c32ec5eab1 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:18:16.669840 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb560ecc-9cfc-41ce-88ba-9fceec441e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:18:16.670490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb560ecc-9cfc-41ce-88ba-9fceec441e97 service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:18:16.670802 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb560ecc-9cfc-41ce-88ba-9fceec441e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:18:16.671074 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb560ecc-9cfc-41ce-88ba-9fceec441e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:18:16.672930 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-cb560ecc-9cfc-41ce-88ba-9fceec441e97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:18:16.687375 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb560ecc-9cfc-41ce-88ba-9fceec441e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:18:16.688849 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb560ecc-9cfc-41ce-88ba-9fceec441e97 service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:18:16.699961 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1461/2920] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:18:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:18:26.526681 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=6 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:18:26.694969 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=12 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:18:37.504515 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67648b19-52d3-46c8-98f7-3728224da7ce service neutron] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:18:37.517351 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67648b19-52d3-46c8-98f7-3728224da7ce service neutron] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:18:37.523352 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2f4cc-b6d5-4fcb-87a1-a7880e6aff3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:18:37.524020 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2f4cc-b6d5-4fcb-87a1-a7880e6aff3b service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:18:37.524309 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2f4cc-b6d5-4fcb-87a1-a7880e6aff3b service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:18:37.524718 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2f4cc-b6d5-4fcb-87a1-a7880e6aff3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:18:37.526587 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-3cc2f4cc-b6d5-4fcb-87a1-a7880e6aff3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:18:37.540028 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2f4cc-b6d5-4fcb-87a1-a7880e6aff3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:18:37.541676 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2f4cc-b6d5-4fcb-87a1-a7880e6aff3b service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:18:37.553542 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1460/2921] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:18:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:18:47.548054 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=6 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:20:07.899490 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb560ecc-9cfc-41ce-88ba-9fceec441e97 service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:20:07.908760 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb560ecc-9cfc-41ce-88ba-9fceec441e97 service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:20:07.912568 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f737339-400f-45eb-8771-af3b711aee1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:20:07.913268 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f737339-400f-45eb-8771-af3b711aee1f service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:20:07.913556 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f737339-400f-45eb-8771-af3b711aee1f service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:20:07.913837 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f737339-400f-45eb-8771-af3b711aee1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:20:07.923416 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f737339-400f-45eb-8771-af3b711aee1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:20:07.924989 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f737339-400f-45eb-8771-af3b711aee1f service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:20:07.935198 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1462/2922] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:20:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:20:11.537171 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2f4cc-b6d5-4fcb-87a1-a7880e6aff3b service placement] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:20:11.553265 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2f4cc-b6d5-4fcb-87a1-a7880e6aff3b service placement] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:20:11.564416 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43e957-b881-4106-9ce3-562c4394d292 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928b8b2b510b4c939b37b2c0dfa57060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:20:11.565272 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43e957-b881-4106-9ce3-562c4394d292 service nova] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:20:11.565740 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43e957-b881-4106-9ce3-562c4394d292 service nova] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:20:11.566217 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43e957-b881-4106-9ce3-562c4394d292 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:20:11.568588 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-aa43e957-b881-4106-9ce3-562c4394d292 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:20:11.589884 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43e957-b881-4106-9ce3-562c4394d292 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:20:11.591951 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43e957-b881-4106-9ce3-562c4394d292 service nova] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:20:11.604554 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1461/2923] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:20:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 21:20:13.034072 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f737339-400f-45eb-8771-af3b711aee1f service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:20:13.045828 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f737339-400f-45eb-8771-af3b711aee1f service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:20:13.049909 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b04c6-4ee8-4895-9fc8-57bc3541868d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5db827b3a66843749e04376559aa27c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:20:13.050602 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b04c6-4ee8-4895-9fc8-57bc3541868d service neutron] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:20:13.050886 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b04c6-4ee8-4895-9fc8-57bc3541868d service neutron] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:20:13.051165 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b04c6-4ee8-4895-9fc8-57bc3541868d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:20:13.123560 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b04c6-4ee8-4895-9fc8-57bc3541868d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:20:13.125064 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b04c6-4ee8-4895-9fc8-57bc3541868d service neutron] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:20:13.135818 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1463/2924] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:20:16.488216 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43e957-b881-4106-9ce3-562c4394d292 service nova] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:20:16.490869 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-aa43e957-b881-4106-9ce3-562c4394d292 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:20:16.529181 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43e957-b881-4106-9ce3-562c4394d292 service nova] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:20:16.543880 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f50b-d90d-4f87-80f8-9b8fd2d0fd8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:20:16.545481 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f50b-d90d-4f87-80f8-9b8fd2d0fd8b service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:20:16.546056 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f50b-d90d-4f87-80f8-9b8fd2d0fd8b service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:20:16.546633 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f50b-d90d-4f87-80f8-9b8fd2d0fd8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:20:16.549601 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-fec1f50b-d90d-4f87-80f8-9b8fd2d0fd8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:20:16.568061 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1f50b-d90d-4f87-80f8-9b8fd2d0fd8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:20:16.571036 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1f50b-d90d-4f87-80f8-9b8fd2d0fd8b service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:20:16.606731 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1462/2925] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:20:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:20:20.147965 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b04c6-4ee8-4895-9fc8-57bc3541868d service neutron] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:20:20.168581 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b04c6-4ee8-4895-9fc8-57bc3541868d service neutron] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:20:20.178190 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20c800-8c7d-4207-9ded-83e207b8d6c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:20:20.179992 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20c800-8c7d-4207-9ded-83e207b8d6c0 service placement] REQUEST_METHOD: `GET` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:20:20.180512 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20c800-8c7d-4207-9ded-83e207b8d6c0 service placement] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:20:20.180942 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20c800-8c7d-4207-9ded-83e207b8d6c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:20:20.184500 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-0b20c800-8c7d-4207-9ded-83e207b8d6c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:20:20.207975 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20c800-8c7d-4207-9ded-83e207b8d6c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:20:20.212067 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20c800-8c7d-4207-9ded-83e207b8d6c0 service placement] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:20:20.228608 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1464/2926] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:20:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:20:20.232630 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f50b-d90d-4f87-80f8-9b8fd2d0fd8b service placement] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:20:20.259348 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f50b-d90d-4f87-80f8-9b8fd2d0fd8b service placement] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:20:20.266978 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:20:20.267883 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:20:20.268165 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:20:20.268491 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] PATH_INFO: `/v3/users/1c61ac31c3a24147ac629bd19f9cb9dc` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:20:20.269805 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c61ac31c3a24147ac629bd19f9cb9dc)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:20:20.271823 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:20:20.329025 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c61ac31c3a24147ac629bd19f9cb9dc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:20.337962 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c61ac31c3a24147ac629bd19f9cb9dc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:20.346940 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c61ac31c3a24147ac629bd19f9cb9dc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:20.347697 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c61ac31c3a24147ac629bd19f9cb9dc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:20.367118 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c61ac31c3a24147ac629bd19f9cb9dc'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:20.377777 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1463/2927] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:20:20 2022] DELETE /identity/v3/users/1c61ac31c3a24147ac629bd19f9cb9dc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:20:20.621816 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20c800-8c7d-4207-9ded-83e207b8d6c0 service placement] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:20:20.644554 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20c800-8c7d-4207-9ded-83e207b8d6c0 service placement] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:20:20.749938 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:20:20.750838 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:20:20.751123 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:20:20.751407 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] PATH_INFO: `/v3/projects/8b9d58fb2dee4edf9632db6ee83312fb` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:20:20.752888 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b9d58fb2dee4edf9632db6ee83312fb)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:20:20.754824 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:20:20.827337 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b9d58fb2dee4edf9632db6ee83312fb', 'request_id': 'req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:20.843510 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b9d58fb2dee4edf9632db6ee83312fb', 'request_id': 'req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:20.851563 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b9d58fb2dee4edf9632db6ee83312fb', 'request_id': 'req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:20.860009 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:20:20.867716 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] The token cache is being invalidate because project 8b9d58fb2dee4edf9632db6ee83312fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:20:20.868111 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:20.869328 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:20.871918 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1465/2928] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:20:20 2022] DELETE /identity/v3/projects/8b9d58fb2dee4edf9632db6ee83312fb => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:20:20.876575 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:20:20.878095 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:20:20.892701 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dafd3-2c54-45c4-94a0-3ba2e7ef9f43 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:20:20.968189 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:20:20.969125 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:20:20.969409 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:20:20.969684 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] PATH_INFO: `/v3/users/e38289ca8c7349f09b246b13d290bf34` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:20:20.971026 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e38289ca8c7349f09b246b13d290bf34)` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:20:20.973043 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:20:21.033845 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e38289ca8c7349f09b246b13d290bf34'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:21.037150 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:20:21.047721 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e38289ca8c7349f09b246b13d290bf34'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:21.058881 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e38289ca8c7349f09b246b13d290bf34'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:21.059692 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e38289ca8c7349f09b246b13d290bf34'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:21.081484 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.notifications [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e38289ca8c7349f09b246b13d290bf34'} {{(pid=86669) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:21.090520 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1464/2929] 10.222.0.85 () {58 vars in 1223 bytes} [Tue Jul 19 21:20:20 2022] DELETE /identity/v3/users/e38289ca8c7349f09b246b13d290bf34 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:20:21.340011 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] Authenticating user token {{(pid=86668) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:20:21.342029 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.fernet_utils [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86668) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:20:21.361144 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692a82d-cd3c-42c5-bdc5-980c01cd6bab admin admin] Validating token access rules against request {{(pid=86668) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:20:21.447566 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adee0d7d3c614774bcfa96ea7aed77e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f322b7a5256429ea77db2c08c676fad', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86668) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:20:21.448513 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:20:21.448807 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:20:21.449117 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] PATH_INFO: `/v3/projects/a4529614c5c64e9eae1ae8e02570e4f9` {{(pid=86668) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:20:21.450453 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4529614c5c64e9eae1ae8e02570e4f9)` {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:20:21.453086 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] RBAC: Authorization granted {{(pid=86668) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:20:21.513142 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4529614c5c64e9eae1ae8e02570e4f9', 'request_id': 'req-585c5622-64ce-4584-a8a9-61d82178bc6d'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:21.523343 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4529614c5c64e9eae1ae8e02570e4f9', 'request_id': 'req-585c5622-64ce-4584-a8a9-61d82178bc6d'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:21.531051 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4529614c5c64e9eae1ae8e02570e4f9', 'request_id': 'req-585c5622-64ce-4584-a8a9-61d82178bc6d'} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:21.543130 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] The token cache is being invalidate because project a4529614c5c64e9eae1ae8e02570e4f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86668) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 21:20:21.543422 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:21.544108 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG keystone.notifications [None req-585c5622-64ce-4584-a8a9-61d82178bc6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86668) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 21:20:21.545765 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: [pid: 86668|app: 0|req: 1466/2930] 10.222.0.85 () {58 vars in 1232 bytes} [Tue Jul 19 21:20:21 2022] DELETE /identity/v3/projects/a4529614c5c64e9eae1ae8e02570e4f9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 21:20:31.084747 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=12 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:20:31.539821 dv-cinder-749155-8-iscsi devstack@keystone.service[86668]: DEBUG dbcounter [-] [86668] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86668) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 21:23:43.455750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] Authenticating user token {{(pid=86669) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 21:23:43.457287 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:23:43.608428 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f714bf3-5c49-4d00-9c60-f04cab9966e6 admin admin] Validating token access rules against request {{(pid=86669) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 21:23:43.671832 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d86fde-3558-41b8-ae1c-7de11872865d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fffebfc51dba47f68263c0bcd6b18d22', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aba35520c454245854855df2e5b42f9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86669) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 21:23:43.673081 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d86fde-3558-41b8-ae1c-7de11872865d service placement] REQUEST_METHOD: `GET` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 21:23:43.673363 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d86fde-3558-41b8-ae1c-7de11872865d service placement] SCRIPT_NAME: `/identity` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 21:23:43.673664 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d86fde-3558-41b8-ae1c-7de11872865d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86669) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 21:23:43.675534 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.fernet_utils [None req-b1d86fde-3558-41b8-ae1c-7de11872865d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86669) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 21:23:43.687548 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d86fde-3558-41b8-ae1c-7de11872865d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 21:23:43.689750 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d86fde-3558-41b8-ae1c-7de11872865d service placement] RBAC: Authorization granted {{(pid=86669) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 21:23:43.700485 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: [pid: 86669|app: 0|req: 1465/2931] 10.222.0.85 () {60 vars in 1365 bytes} [Tue Jul 19 21:23:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 21:23:53.695682 dv-cinder-749155-8-iscsi devstack@keystone.service[86669]: DEBUG dbcounter [-] [86669] Writing DB stats keystone:SELECT=89 {{(pid=86669) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}